Locally Private Bayesian Inference for Count Models · Poisson factorization of the non-privatized data, suggesting that non-private Poisson factorization may be overﬁtting. 2

Locally Private Bayesian Inference for Count Models

Aaron Schein 1 Zhiwei Steven Wu 2 Alexandra Schofield 3 Mingyuan Zhou 4 Hanna Wallach 5

AbstractWe present a general and modular method forprivacy-preserving Bayesian inference for Pois-son factorization, a broad class of models thatincludes some of the most widely used models inthe social sciences. Our method satisfies limited-precision local privacy, a generalization of localdifferential privacy that we introduce to formulateappropriate privacy guarantees for sparse countdata. We present an MCMC algorithm that ap-proximates the posterior distribution over the la-tent variables conditioned on data that has beenlocally privatized by the geometric mechanism.Our method is based on two insights: 1) a novelreinterpretation of the geometric mechanism interms of the Skellam distribution and 2) a gen-eral theorem that relates the Skellam and Besseldistributions. We demonstrate our method’s util-ity using two case studies that involve real-worldemail data. We show that our method consistentlyoutperforms the commonly used naıve approach,wherein inference proceeds as usual, treating thelocally privatized data as if it were not privatized.

1. IntroductionData from social processes often take the form of discreteobservations (e.g., edges in a social network, word tokensin an email). These observations often contain sensitiveinformation. As more aspects of social interaction are digi-tally recorded, the opportunities for social scientific insightsgrow; however, so too does the risk of unacceptable privacyviolations. As a result, there is a growing need to developprivacy-preserving data analysis methods. In practice, socialscientists will be more likely to adopt these methods if doingso entails minimal change to their current methodology.

1University of Massachusetts Amherst 2University of Min-nesota 3Cornell University 4University of Texas at Austin5Microsoft. Correspondence to: Aaron Schein <[email protected]>.

Proceedings of the 36 th International Conference on MachineLearning, Long Beach, California, PMLR 97, 2019. Copyright2019 by the author(s).

Toward that end, we present a method for privacy-preservingBayesian inference for Poisson factorization (Titsias, 2008;Cemgil, 2009; Zhou & Carin, 2012; Gopalan & Blei, 2013;Paisley et al., 2014), a broad class of models for inferringlatent structure from discrete data. This class contains someof the most widely used models in the social sciences, in-cluding topic models for text corpora (Blei et al., 2003;Buntine & Jakulin, 2004; Canny, 2004), population modelsfor genetic data (Pritchard et al., 2000), stochastic blockmodels for social networks (Ball et al., 2011; Gopalan &Blei, 2013; Zhou, 2015), and tensor factorization of dyadicdata (Welling & Weber, 2001; Chi & Kolda, 2012; Schmidt& Morup, 2013; Schein et al., 2015; 2016b). It further in-cludes deep hierarchical models (Ranganath et al., 2015;Zhou et al., 2015), dynamic models (Charlin et al., 2015;Acharya et al., 2015; Schein et al., 2016a), and many others.

Our method assumes that observations are privatized (ornoised) via a randomized response method before they areaggregated into a data set for analysis. This ensures thatno single location need ever store the non-privatized dataset. We introduce limited-precision local privacy (LPLP)—ageneralization of local differential privacy—in order to for-mulate appropriate privacy guarantees for sparse count data.We focus specifically on the geometric mechanism of Ghoshet al. (2012) and prove that it is a mechanism for LPLP.

Under local privacy, a data analysis algorithm sees onlythe privatized data set. Inferring latent structure (includingmodel parameters) that accurately reflects the non-privatizeddata set is therefore a key statistical challenge. One optionis a naıve approach, wherein inference proceeds as usual,treating the privatized data set as if it were not privatized.In the context of maximum likelihood estimation, the naıveapproach has been shown to exhibit pathologies whenobservations are discrete or count-valued. Researchershave therefore advocated for treating the non-privatizedobservations as latent variables to be inferred (Yang et al.,2012; Karwa et al., 2014; Bernstein et al., 2017; Bernstein &Sheldon, 2018). We embrace this approach and extend it toBayesian inference for Poisson factorization, where our goalis to approximate the locally private posterior distributionover the latent variables conditioned on the privatized dataset and the randomized response method. Toward thatgoal, we present a Markov chain Monte Carlo (MCMC)algorithm that is asymptotically guaranteed to draw samples


True data yij

True µ⇤ij

Non-private µij Our estimate µij

Naıve estimate µij

Noised data y(±)ij

Increasing privacy ✏/N =1<latexit sha1_base64="j3BuK7EHK7lxhCJ2ICD/7fEl4Ms=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM7UIXdVEBN0IBTeupIJ9QBPKZDppByeTMDMRaij+ihsXirj1P9z5N07bLLT1wIXDOfdy7z1BwpnSjvNtFZaWV1bXiuuljc2t7R17d6+l4lQS2iQxj2UnwIpyJmhTM81pJ5EURwGn7eD+auK3H6hULBZ3epRQP8IDwUJGsDZSzz7waKIYjwU6QTfIK196ZeT27IpTc6ZAi8TNSQVyNHr2l9ePSRpRoQnHSnVdJ9F+hqVmhNNxyUsVTTC5xwPaNVTgiCo/m14/RsdG6aMwlqaERlP190SGI6VGUWA6I6yHat6biP953VSHF37GRJJqKshsUZhypGM0iQL1maRE85EhmEhmbkVkiCUm2gRWMiG48y8vktZpzXVq7u1ZpV7N4yjCIRxBFVw4hzpcQwOaQOARnuEV3qwn68V6tz5mrQUrn9mHP7A+fwBPfpMU</latexit><latexit sha1_base64="j3BuK7EHK7lxhCJ2ICD/7fEl4Ms=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM7UIXdVEBN0IBTeupIJ9QBPKZDppByeTMDMRaij+ihsXirj1P9z5N07bLLT1wIXDOfdy7z1BwpnSjvNtFZaWV1bXiuuljc2t7R17d6+l4lQS2iQxj2UnwIpyJmhTM81pJ5EURwGn7eD+auK3H6hULBZ3epRQP8IDwUJGsDZSzz7waKIYjwU6QTfIK196ZeT27IpTc6ZAi8TNSQVyNHr2l9ePSRpRoQnHSnVdJ9F+hqVmhNNxyUsVTTC5xwPaNVTgiCo/m14/RsdG6aMwlqaERlP190SGI6VGUWA6I6yHat6biP953VSHF37GRJJqKshsUZhypGM0iQL1maRE85EhmEhmbkVkiCUm2gRWMiG48y8vktZpzXVq7u1ZpV7N4yjCIRxBFVw4hzpcQwOaQOARnuEV3qwn68V6tz5mrQUrn9mHP7A+fwBPfpMU</latexit><latexit sha1_base64="j3BuK7EHK7lxhCJ2ICD/7fEl4Ms=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM7UIXdVEBN0IBTeupIJ9QBPKZDppByeTMDMRaij+ihsXirj1P9z5N07bLLT1wIXDOfdy7z1BwpnSjvNtFZaWV1bXiuuljc2t7R17d6+l4lQS2iQxj2UnwIpyJmhTM81pJ5EURwGn7eD+auK3H6hULBZ3epRQP8IDwUJGsDZSzz7waKIYjwU6QTfIK196ZeT27IpTc6ZAi8TNSQVyNHr2l9ePSRpRoQnHSnVdJ9F+hqVmhNNxyUsVTTC5xwPaNVTgiCo/m14/RsdG6aMwlqaERlP190SGI6VGUWA6I6yHat6biP953VSHF37GRJJqKshsUZhypGM0iQL1maRE85EhmEhmbkVkiCUm2gRWMiG48y8vktZpzXVq7u1ZpV7N4yjCIRxBFVw4hzpcQwOaQOARnuEV3qwn68V6tz5mrQUrn9mHP7A+fwBPfpMU</latexit><latexit sha1_base64="j3BuK7EHK7lxhCJ2ICD/7fEl4Ms=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM7UIXdVEBN0IBTeupIJ9QBPKZDppByeTMDMRaij+ihsXirj1P9z5N07bLLT1wIXDOfdy7z1BwpnSjvNtFZaWV1bXiuuljc2t7R17d6+l4lQS2iQxj2UnwIpyJmhTM81pJ5EURwGn7eD+auK3H6hULBZ3epRQP8IDwUJGsDZSzz7waKIYjwU6QTfIK196ZeT27IpTc6ZAi8TNSQVyNHr2l9ePSRpRoQnHSnVdJ9F+hqVmhNNxyUsVTTC5xwPaNVTgiCo/m14/RsdG6aMwlqaERlP190SGI6VGUWA6I6yHat6biP953VSHF37GRJJqKshsUZhypGM0iQL1maRE85EhmEhmbkVkiCUm2gRWMiG48y8vktZpzXVq7u1ZpV7N4yjCIRxBFVw4hzpcQwOaQOARnuEV3qwn68V6tz5mrQUrn9mHP7A+fwBPfpMU</latexit>

✏/N =0.5<latexit sha1_base64="Ng0gtFtWYod03th7XEOH2TNuvkg=">AAAB/XicbVDLSsNAFJ3UV62v+Ni5mVqErmIiim6EghtXUsE+oAllMp20QyczYWYi1FD8FTcuFHHrf7jzb5y2WWjrgQuHc+7l3nvChFGlXffbKiwtr6yuFddLG5tb2zv27l5TiVRi0sCCCdkOkSKMctLQVDPSTiRBcchIKxxeT/zWA5GKCn6vRwkJYtTnNKIYaSN17QOfJIoywU9uoV++8svQdc67dsV13CngIvFyUgE56l37y+8JnMaEa8yQUh3PTXSQIakpZmRc8lNFEoSHqE86hnIUExVk0+vH8NgoPRgJaYprOFV/T2QoVmoUh6YzRnqg5r2J+J/XSXV0GWSUJ6kmHM8WRSmDWsBJFLBHJcGajQxBWFJzK8QDJBHWJrCSCcGbf3mRNE8dz3W8u7NKrZrHUQSH4AhUgQcuQA3cgDpoAAwewTN4BW/Wk/VivVsfs9aClc/sgz+wPn8AhM2TNg==</latexit><latexit sha1_base64="Ng0gtFtWYod03th7XEOH2TNuvkg=">AAAB/XicbVDLSsNAFJ3UV62v+Ni5mVqErmIiim6EghtXUsE+oAllMp20QyczYWYi1FD8FTcuFHHrf7jzb5y2WWjrgQuHc+7l3nvChFGlXffbKiwtr6yuFddLG5tb2zv27l5TiVRi0sCCCdkOkSKMctLQVDPSTiRBcchIKxxeT/zWA5GKCn6vRwkJYtTnNKIYaSN17QOfJIoywU9uoV++8svQdc67dsV13CngIvFyUgE56l37y+8JnMaEa8yQUh3PTXSQIakpZmRc8lNFEoSHqE86hnIUExVk0+vH8NgoPRgJaYprOFV/T2QoVmoUh6YzRnqg5r2J+J/XSXV0GWSUJ6kmHM8WRSmDWsBJFLBHJcGajQxBWFJzK8QDJBHWJrCSCcGbf3mRNE8dz3W8u7NKrZrHUQSH4AhUgQcuQA3cgDpoAAwewTN4BW/Wk/VivVsfs9aClc/sgz+wPn8AhM2TNg==</latexit><latexit sha1_base64="Ng0gtFtWYod03th7XEOH2TNuvkg=">AAAB/XicbVDLSsNAFJ3UV62v+Ni5mVqErmIiim6EghtXUsE+oAllMp20QyczYWYi1FD8FTcuFHHrf7jzb5y2WWjrgQuHc+7l3nvChFGlXffbKiwtr6yuFddLG5tb2zv27l5TiVRi0sCCCdkOkSKMctLQVDPSTiRBcchIKxxeT/zWA5GKCn6vRwkJYtTnNKIYaSN17QOfJIoywU9uoV++8svQdc67dsV13CngIvFyUgE56l37y+8JnMaEa8yQUh3PTXSQIakpZmRc8lNFEoSHqE86hnIUExVk0+vH8NgoPRgJaYprOFV/T2QoVmoUh6YzRnqg5r2J+J/XSXV0GWSUJ6kmHM8WRSmDWsBJFLBHJcGajQxBWFJzK8QDJBHWJrCSCcGbf3mRNE8dz3W8u7NKrZrHUQSH4AhUgQcuQA3cgDpoAAwewTN4BW/Wk/VivVsfs9aClc/sgz+wPn8AhM2TNg==</latexit><latexit sha1_base64="Ng0gtFtWYod03th7XEOH2TNuvkg=">AAAB/XicbVDLSsNAFJ3UV62v+Ni5mVqErmIiim6EghtXUsE+oAllMp20QyczYWYi1FD8FTcuFHHrf7jzb5y2WWjrgQuHc+7l3nvChFGlXffbKiwtr6yuFddLG5tb2zv27l5TiVRi0sCCCdkOkSKMctLQVDPSTiRBcchIKxxeT/zWA5GKCn6vRwkJYtTnNKIYaSN17QOfJIoywU9uoV++8svQdc67dsV13CngIvFyUgE56l37y+8JnMaEa8yQUh3PTXSQIakpZmRc8lNFEoSHqE86hnIUExVk0+vH8NgoPRgJaYprOFV/T2QoVmoUh6YzRnqg5r2J+J/XSXV0GWSUJ6kmHM8WRSmDWsBJFLBHJcGajQxBWFJzK8QDJBHWJrCSCcGbf3mRNE8dz3W8u7NKrZrHUQSH4AhUgQcuQA3cgDpoAAwewTN4BW/Wk/VivVsfs9aClc/sgz+wPn8AhM2TNg==</latexit>

✏/N =0.25<latexit sha1_base64="P8JoUQIFhMtPZEZq/fLaV9Xkt6E=">AAAB/nicbVDLSsNAFJ3UV62vqLhyM7UIXcWkKLoRCm5cSQX7gCaUyXTSDp3MhJmJUELBX3HjQhG3foc7/8Zpm4W2HrhwOOde7r0nTBhV2nW/rcLK6tr6RnGztLW9s7tn7x+0lEglJk0smJCdECnCKCdNTTUjnUQSFIeMtMPRzdRvPxKpqOAPepyQIEYDTiOKkTZSzz7ySaIoE/zsDvrla78MXad20bMrruPOAJeJl5MKyNHo2V9+X+A0JlxjhpTqem6igwxJTTEjk5KfKpIgPEID0jWUo5ioIJudP4GnRunDSEhTXMOZ+nsiQ7FS4zg0nTHSQ7XoTcX/vG6qo6sgozxJNeF4vihKGdQCTrOAfSoJ1mxsCMKSmlshHiKJsDaJlUwI3uLLy6RVczzX8e7PK/VqHkcRHIMTUAUeuAR1cAsaoAkwyMAzeAVv1pP1Yr1bH/PWgpXPHII/sD5/APuYk3I=</latexit><latexit sha1_base64="P8JoUQIFhMtPZEZq/fLaV9Xkt6E=">AAAB/nicbVDLSsNAFJ3UV62vqLhyM7UIXcWkKLoRCm5cSQX7gCaUyXTSDp3MhJmJUELBX3HjQhG3foc7/8Zpm4W2HrhwOOde7r0nTBhV2nW/rcLK6tr6RnGztLW9s7tn7x+0lEglJk0smJCdECnCKCdNTTUjnUQSFIeMtMPRzdRvPxKpqOAPepyQIEYDTiOKkTZSzz7ySaIoE/zsDvrla78MXad20bMrruPOAJeJl5MKyNHo2V9+X+A0JlxjhpTqem6igwxJTTEjk5KfKpIgPEID0jWUo5ioIJudP4GnRunDSEhTXMOZ+nsiQ7FS4zg0nTHSQ7XoTcX/vG6qo6sgozxJNeF4vihKGdQCTrOAfSoJ1mxsCMKSmlshHiKJsDaJlUwI3uLLy6RVczzX8e7PK/VqHkcRHIMTUAUeuAR1cAsaoAkwyMAzeAVv1pP1Yr1bH/PWgpXPHII/sD5/APuYk3I=</latexit><latexit sha1_base64="P8JoUQIFhMtPZEZq/fLaV9Xkt6E=">AAAB/nicbVDLSsNAFJ3UV62vqLhyM7UIXcWkKLoRCm5cSQX7gCaUyXTSDp3MhJmJUELBX3HjQhG3foc7/8Zpm4W2HrhwOOde7r0nTBhV2nW/rcLK6tr6RnGztLW9s7tn7x+0lEglJk0smJCdECnCKCdNTTUjnUQSFIeMtMPRzdRvPxKpqOAPepyQIEYDTiOKkTZSzz7ySaIoE/zsDvrla78MXad20bMrruPOAJeJl5MKyNHo2V9+X+A0JlxjhpTqem6igwxJTTEjk5KfKpIgPEID0jWUo5ioIJudP4GnRunDSEhTXMOZ+nsiQ7FS4zg0nTHSQ7XoTcX/vG6qo6sgozxJNeF4vihKGdQCTrOAfSoJ1mxsCMKSmlshHiKJsDaJlUwI3uLLy6RVczzX8e7PK/VqHkcRHIMTUAUeuAR1cAsaoAkwyMAzeAVv1pP1Yr1bH/PWgpXPHII/sD5/APuYk3I=</latexit><latexit sha1_base64="P8JoUQIFhMtPZEZq/fLaV9Xkt6E=">AAAB/nicbVDLSsNAFJ3UV62vqLhyM7UIXcWkKLoRCm5cSQX7gCaUyXTSDp3MhJmJUELBX3HjQhG3foc7/8Zpm4W2HrhwOOde7r0nTBhV2nW/rcLK6tr6RnGztLW9s7tn7x+0lEglJk0smJCdECnCKCdNTTUjnUQSFIeMtMPRzdRvPxKpqOAPepyQIEYDTiOKkTZSzz7ySaIoE/zsDvrla78MXad20bMrruPOAJeJl5MKyNHo2V9+X+A0JlxjhpTqem6igwxJTTEjk5KfKpIgPEID0jWUo5ioIJudP4GnRunDSEhTXMOZ+nsiQ7FS4zg0nTHSQ7XoTcX/vG6qo6sgozxJNeF4vihKGdQCTrOAfSoJ1mxsCMKSmlshHiKJsDaJlUwI3uLLy6RVczzX8e7PK/VqHkcRHIMTUAUeuAR1cAsaoAkwyMAzeAVv1pP1Yr1bH/PWgpXPHII/sD5/APuYk3I=</latexit>

True data yij

True µ⇤ij

Non-private µij Our estimate µij

Naıve estimate µij

Noised data y(±)ij

Increasing privacy ✏/N =1<latexit sha1_base64="j3BuK7EHK7lxhCJ2ICD/7fEl4Ms=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM7UIXdVEBN0IBTeupIJ9QBPKZDppByeTMDMRaij+ihsXirj1P9z5N07bLLT1wIXDOfdy7z1BwpnSjvNtFZaWV1bXiuuljc2t7R17d6+l4lQS2iQxj2UnwIpyJmhTM81pJ5EURwGn7eD+auK3H6hULBZ3epRQP8IDwUJGsDZSzz7waKIYjwU6QTfIK196ZeT27IpTc6ZAi8TNSQVyNHr2l9ePSRpRoQnHSnVdJ9F+hqVmhNNxyUsVTTC5xwPaNVTgiCo/m14/RsdG6aMwlqaERlP190SGI6VGUWA6I6yHat6biP953VSHF37GRJJqKshsUZhypGM0iQL1maRE85EhmEhmbkVkiCUm2gRWMiG48y8vktZpzXVq7u1ZpV7N4yjCIRxBFVw4hzpcQwOaQOARnuEV3qwn68V6tz5mrQUrn9mHP7A+fwBPfpMU</latexit><latexit sha1_base64="j3BuK7EHK7lxhCJ2ICD/7fEl4Ms=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM7UIXdVEBN0IBTeupIJ9QBPKZDppByeTMDMRaij+ihsXirj1P9z5N07bLLT1wIXDOfdy7z1BwpnSjvNtFZaWV1bXiuuljc2t7R17d6+l4lQS2iQxj2UnwIpyJmhTM81pJ5EURwGn7eD+auK3H6hULBZ3epRQP8IDwUJGsDZSzz7waKIYjwU6QTfIK196ZeT27IpTc6ZAi8TNSQVyNHr2l9ePSRpRoQnHSnVdJ9F+hqVmhNNxyUsVTTC5xwPaNVTgiCo/m14/RsdG6aMwlqaERlP190SGI6VGUWA6I6yHat6biP953VSHF37GRJJqKshsUZhypGM0iQL1maRE85EhmEhmbkVkiCUm2gRWMiG48y8vktZpzXVq7u1ZpV7N4yjCIRxBFVw4hzpcQwOaQOARnuEV3qwn68V6tz5mrQUrn9mHP7A+fwBPfpMU</latexit><latexit sha1_base64="j3BuK7EHK7lxhCJ2ICD/7fEl4Ms=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM7UIXdVEBN0IBTeupIJ9QBPKZDppByeTMDMRaij+ihsXirj1P9z5N07bLLT1wIXDOfdy7z1BwpnSjvNtFZaWV1bXiuuljc2t7R17d6+l4lQS2iQxj2UnwIpyJmhTM81pJ5EURwGn7eD+auK3H6hULBZ3epRQP8IDwUJGsDZSzz7waKIYjwU6QTfIK196ZeT27IpTc6ZAi8TNSQVyNHr2l9ePSRpRoQnHSnVdJ9F+hqVmhNNxyUsVTTC5xwPaNVTgiCo/m14/RsdG6aMwlqaERlP190SGI6VGUWA6I6yHat6biP953VSHF37GRJJqKshsUZhypGM0iQL1maRE85EhmEhmbkVkiCUm2gRWMiG48y8vktZpzXVq7u1ZpV7N4yjCIRxBFVw4hzpcQwOaQOARnuEV3qwn68V6tz5mrQUrn9mHP7A+fwBPfpMU</latexit><latexit sha1_base64="j3BuK7EHK7lxhCJ2ICD/7fEl4Ms=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM7UIXdVEBN0IBTeupIJ9QBPKZDppByeTMDMRaij+ihsXirj1P9z5N07bLLT1wIXDOfdy7z1BwpnSjvNtFZaWV1bXiuuljc2t7R17d6+l4lQS2iQxj2UnwIpyJmhTM81pJ5EURwGn7eD+auK3H6hULBZ3epRQP8IDwUJGsDZSzz7waKIYjwU6QTfIK196ZeT27IpTc6ZAi8TNSQVyNHr2l9ePSRpRoQnHSnVdJ9F+hqVmhNNxyUsVTTC5xwPaNVTgiCo/m14/RsdG6aMwlqaERlP190SGI6VGUWA6I6yHat6biP953VSHF37GRJJqKshsUZhypGM0iQL1maRE85EhmEhmbkVkiCUm2gRWMiG48y8vktZpzXVq7u1ZpV7N4yjCIRxBFVw4hzpcQwOaQOARnuEV3qwn68V6tz5mrQUrn9mHP7A+fwBPfpMU</latexit>

✏/N =0.5<latexit sha1_base64="Ng0gtFtWYod03th7XEOH2TNuvkg=">AAAB/XicbVDLSsNAFJ3UV62v+Ni5mVqErmIiim6EghtXUsE+oAllMp20QyczYWYi1FD8FTcuFHHrf7jzb5y2WWjrgQuHc+7l3nvChFGlXffbKiwtr6yuFddLG5tb2zv27l5TiVRi0sCCCdkOkSKMctLQVDPSTiRBcchIKxxeT/zWA5GKCn6vRwkJYtTnNKIYaSN17QOfJIoywU9uoV++8svQdc67dsV13CngIvFyUgE56l37y+8JnMaEa8yQUh3PTXSQIakpZmRc8lNFEoSHqE86hnIUExVk0+vH8NgoPRgJaYprOFV/T2QoVmoUh6YzRnqg5r2J+J/XSXV0GWSUJ6kmHM8WRSmDWsBJFLBHJcGajQxBWFJzK8QDJBHWJrCSCcGbf3mRNE8dz3W8u7NKrZrHUQSH4AhUgQcuQA3cgDpoAAwewTN4BW/Wk/VivVsfs9aClc/sgz+wPn8AhM2TNg==</latexit><latexit sha1_base64="Ng0gtFtWYod03th7XEOH2TNuvkg=">AAAB/XicbVDLSsNAFJ3UV62v+Ni5mVqErmIiim6EghtXUsE+oAllMp20QyczYWYi1FD8FTcuFHHrf7jzb5y2WWjrgQuHc+7l3nvChFGlXffbKiwtr6yuFddLG5tb2zv27l5TiVRi0sCCCdkOkSKMctLQVDPSTiRBcchIKxxeT/zWA5GKCn6vRwkJYtTnNKIYaSN17QOfJIoywU9uoV++8svQdc67dsV13CngIvFyUgE56l37y+8JnMaEa8yQUh3PTXSQIakpZmRc8lNFEoSHqE86hnIUExVk0+vH8NgoPRgJaYprOFV/T2QoVmoUh6YzRnqg5r2J+J/XSXV0GWSUJ6kmHM8WRSmDWsBJFLBHJcGajQxBWFJzK8QDJBHWJrCSCcGbf3mRNE8dz3W8u7NKrZrHUQSH4AhUgQcuQA3cgDpoAAwewTN4BW/Wk/VivVsfs9aClc/sgz+wPn8AhM2TNg==</latexit><latexit sha1_base64="Ng0gtFtWYod03th7XEOH2TNuvkg=">AAAB/XicbVDLSsNAFJ3UV62v+Ni5mVqErmIiim6EghtXUsE+oAllMp20QyczYWYi1FD8FTcuFHHrf7jzb5y2WWjrgQuHc+7l3nvChFGlXffbKiwtr6yuFddLG5tb2zv27l5TiVRi0sCCCdkOkSKMctLQVDPSTiRBcchIKxxeT/zWA5GKCn6vRwkJYtTnNKIYaSN17QOfJIoywU9uoV++8svQdc67dsV13CngIvFyUgE56l37y+8JnMaEa8yQUh3PTXSQIakpZmRc8lNFEoSHqE86hnIUExVk0+vH8NgoPRgJaYprOFV/T2QoVmoUh6YzRnqg5r2J+J/XSXV0GWSUJ6kmHM8WRSmDWsBJFLBHJcGajQxBWFJzK8QDJBHWJrCSCcGbf3mRNE8dz3W8u7NKrZrHUQSH4AhUgQcuQA3cgDpoAAwewTN4BW/Wk/VivVsfs9aClc/sgz+wPn8AhM2TNg==</latexit><latexit sha1_base64="Ng0gtFtWYod03th7XEOH2TNuvkg=">AAAB/XicbVDLSsNAFJ3UV62v+Ni5mVqErmIiim6EghtXUsE+oAllMp20QyczYWYi1FD8FTcuFHHrf7jzb5y2WWjrgQuHc+7l3nvChFGlXffbKiwtr6yuFddLG5tb2zv27l5TiVRi0sCCCdkOkSKMctLQVDPSTiRBcchIKxxeT/zWA5GKCn6vRwkJYtTnNKIYaSN17QOfJIoywU9uoV++8svQdc67dsV13CngIvFyUgE56l37y+8JnMaEa8yQUh3PTXSQIakpZmRc8lNFEoSHqE86hnIUExVk0+vH8NgoPRgJaYprOFV/T2QoVmoUh6YzRnqg5r2J+J/XSXV0GWSUJ6kmHM8WRSmDWsBJFLBHJcGajQxBWFJzK8QDJBHWJrCSCcGbf3mRNE8dz3W8u7NKrZrHUQSH4AhUgQcuQA3cgDpoAAwewTN4BW/Wk/VivVsfs9aClc/sgz+wPn8AhM2TNg==</latexit>

✏/N =0.25<latexit sha1_base64="P8JoUQIFhMtPZEZq/fLaV9Xkt6E=">AAAB/nicbVDLSsNAFJ3UV62vqLhyM7UIXcWkKLoRCm5cSQX7gCaUyXTSDp3MhJmJUELBX3HjQhG3foc7/8Zpm4W2HrhwOOde7r0nTBhV2nW/rcLK6tr6RnGztLW9s7tn7x+0lEglJk0smJCdECnCKCdNTTUjnUQSFIeMtMPRzdRvPxKpqOAPepyQIEYDTiOKkTZSzz7ySaIoE/zsDvrla78MXad20bMrruPOAJeJl5MKyNHo2V9+X+A0JlxjhpTqem6igwxJTTEjk5KfKpIgPEID0jWUo5ioIJudP4GnRunDSEhTXMOZ+nsiQ7FS4zg0nTHSQ7XoTcX/vG6qo6sgozxJNeF4vihKGdQCTrOAfSoJ1mxsCMKSmlshHiKJsDaJlUwI3uLLy6RVczzX8e7PK/VqHkcRHIMTUAUeuAR1cAsaoAkwyMAzeAVv1pP1Yr1bH/PWgpXPHII/sD5/APuYk3I=</latexit><latexit sha1_base64="P8JoUQIFhMtPZEZq/fLaV9Xkt6E=">AAAB/nicbVDLSsNAFJ3UV62vqLhyM7UIXcWkKLoRCm5cSQX7gCaUyXTSDp3MhJmJUELBX3HjQhG3foc7/8Zpm4W2HrhwOOde7r0nTBhV2nW/rcLK6tr6RnGztLW9s7tn7x+0lEglJk0smJCdECnCKCdNTTUjnUQSFIeMtMPRzdRvPxKpqOAPepyQIEYDTiOKkTZSzz7ySaIoE/zsDvrla78MXad20bMrruPOAJeJl5MKyNHo2V9+X+A0JlxjhpTqem6igwxJTTEjk5KfKpIgPEID0jWUo5ioIJudP4GnRunDSEhTXMOZ+nsiQ7FS4zg0nTHSQ7XoTcX/vG6qo6sgozxJNeF4vihKGdQCTrOAfSoJ1mxsCMKSmlshHiKJsDaJlUwI3uLLy6RVczzX8e7PK/VqHkcRHIMTUAUeuAR1cAsaoAkwyMAzeAVv1pP1Yr1bH/PWgpXPHII/sD5/APuYk3I=</latexit><latexit sha1_base64="P8JoUQIFhMtPZEZq/fLaV9Xkt6E=">AAAB/nicbVDLSsNAFJ3UV62vqLhyM7UIXcWkKLoRCm5cSQX7gCaUyXTSDp3MhJmJUELBX3HjQhG3foc7/8Zpm4W2HrhwOOde7r0nTBhV2nW/rcLK6tr6RnGztLW9s7tn7x+0lEglJk0smJCdECnCKCdNTTUjnUQSFIeMtMPRzdRvPxKpqOAPepyQIEYDTiOKkTZSzz7ySaIoE/zsDvrla78MXad20bMrruPOAJeJl5MKyNHo2V9+X+A0JlxjhpTqem6igwxJTTEjk5KfKpIgPEID0jWUo5ioIJudP4GnRunDSEhTXMOZ+nsiQ7FS4zg0nTHSQ7XoTcX/vG6qo6sgozxJNeF4vihKGdQCTrOAfSoJ1mxsCMKSmlshHiKJsDaJlUwI3uLLy6RVczzX8e7PK/VqHkcRHIMTUAUeuAR1cAsaoAkwyMAzeAVv1pP1Yr1bH/PWgpXPHII/sD5/APuYk3I=</latexit><latexit sha1_base64="P8JoUQIFhMtPZEZq/fLaV9Xkt6E=">AAAB/nicbVDLSsNAFJ3UV62vqLhyM7UIXcWkKLoRCm5cSQX7gCaUyXTSDp3MhJmJUELBX3HjQhG3foc7/8Zpm4W2HrhwOOde7r0nTBhV2nW/rcLK6tr6RnGztLW9s7tn7x+0lEglJk0smJCdECnCKCdNTTUjnUQSFIeMtMPRzdRvPxKpqOAPepyQIEYDTiOKkTZSzz7ySaIoE/zsDvrla78MXad20bMrruPOAJeJl5MKyNHo2V9+X+A0JlxjhpTqem6igwxJTTEjk5KfKpIgPEID0jWUo5ioIJudP4GnRunDSEhTXMOZ+nsiQ7FS4zg0nTHSQ7XoTcX/vG6qo6sgozxJNeF4vihKGdQCTrOAfSoJ1mxsCMKSmlshHiKJsDaJlUwI3uLLy6RVczzX8e7PK/VqHkcRHIMTUAUeuAR1cAsaoAkwyMAzeAVv1pP1Yr1bH/PWgpXPHII/sD5/APuYk3I=</latexit>

Figure 1. Block structure recovery: our method vs. the naıve approach. We generated the non-privatized data set synthetically. We thenprivatized the data set using three levels of noise. The top row depicts the data set, using red to denote positive counts and blue to denotenegative counts. As the privacy level increases, the naıve approach overfits the noise and fails to recover the latent structure µ?

ij , predictinghigh values even for sparse parts of the matrix. In contrast, our method recovers the latent structure, even for high levels of privacy.

from the locally private posterior. This algorithm is modular,allowing social scientists to extend (rather than replace)their implementations of non-private Poisson factorization.

Our main technical contribution is the derivation of a closed-form, computationally tractable way of “inverting” the ran-domized response method—i.e., sampling values of thenon-privatized data set from its complete conditional. Thisderivation relies on two insights: 1) a novel reinterpreta-tion of the geometric mechanism in terms of the Skellamdistribution (Skellam, 1946) and 2) a general theorem thatrelates the Skellam and Bessel (Yuan & Kalbfleisch, 2000)distributions. These insights may be of independent interest.

We present two case studies applying our method to 1) topicmodeling for text corpora and 2) overlapping communitydetection for social networks. Using real-world data fromthe Enron email corpus (Klimt & Yang, 2004), we show thatour method consistently outperforms the naıve approachaccording to a variety of intrinsic and extrinsic evaluationmetrics. We provide an illustrative example in figure 1.Finally, we note that our method sometimes outperformsPoisson factorization of the non-privatized data, suggestingthat non-private Poisson factorization may be overfitting.

2. Background and problem formulationDifferential privacy. Differential privacy (Dwork et al.,2006) is a rigorous privacy criterion that guarantees that nosingle observation in a data set will have a significant influ-ence on the information obtained by analyzing that data set.

Definition 1. A randomized algorithm A(·) satisfies ε-differential privacy if for all pairs of neighboring data setsY and Y ′ that differ in only a single observation

P (A(Y ) ∈ S) ≤ eεP (A(Y ′) ∈ S) , (1)

for all subsets S in the range of A(·).

Local differential privacy. We focus on local differentialprivacy, which we refer to as local privacy. Under this crite-rion, the observations remain private from even the data anal-ysis algorithm. The algorithm sees only privatized versionsof the observations, constructed by adding noise from spe-cific distributions. The process of adding noise is known asrandomized response—a reference to survey-sampling meth-ods originally developed in the social sciences prior to thedevelopment of differential privacy (Warner, 1965). Satisfy-ing this criterion means that no single location (e.g., a cen-tralized server) need ever store the non-privatized data set.

Definition 2. A randomized response method R(·) isε-private if for all pairs of observations y, y′ ∈ Y

P (R(y) ∈ S) ≤ eεP (R(y′) ∈ S) , (2)

for all subsets S in the range of R(·). If a data analysisalgorithm sees only the observations’ ε-private responses,then the data analysis itself satisfies ε-local privacy.

The meaning of “observation” in definitions 1 and 2 variesdepending on the context. For example, in the context oftopic modeling, an observation is an entire document—i.e.,a vector of counts representing the number of times eachword type occurs in that document. To guarantee local


privacy, a randomized response method must satisfy thecondition in equation 2 for all pairs of observations. Thistypically involves adding noise that scales with the maxi-mum difference between any pair of observations N (max) =maxy,y′ ‖y−y′‖1. When an observation is a document,N (max) can be prohibitively large, meaning that the noisecan overwhelm the signal in the data set. This challenge mo-tivates the following alternative formulation of local privacy.

Limited-precision local privacy. Local privacy requiresthat a randomized response method render indistinguishablepairs of observations that are arbitrarily different. In somecontexts, this requirement is unnecessarily strong. Forexample, the author of a document may only wish to concealthe occurrence of a handful of word types. To achieve thisgoal, a randomized response method need only render indis-tinguishable pairs of similar documents, such as a documentin which those word types occur and an otherwise-identicaldocument in which they do not. We operationalize this kindof privacy guarantee by generalizing definition 2 as follows.

Definition 3. For any positive integer N , we say that arandomized response method R(·) is (N, ε)-private if forall pairs of observations y, y′ ∈ Y such that ‖y−y′‖1 ≤ N

P (R(y) ∈ S) ≤ eεP (R(y′) ∈ S) , (3)

for all subsets S in the range ofR(·). If a data analysis algo-rithm sees only the observations’ (N, ε)-private responses,then the data analysis itself satisfies (N, ε)-limited-precisionlocal privacy. If ‖y‖1 ≤ N for all y ∈ Y , then (N, ε)-limited-precision local privacy implies ε-local privacy.

Limited-precision local privacy (LPLP) is the local privacyanalog of limited-precision differential privacy, originallyproposed by Flood et al. (2013) and subsequently usedto privatize analyses of geographic location data (Andreset al., 2013) and financial network data (Papadimitriou et al.,2017). Like profile-based privacy (Geumlek & Chaudhuri,2019), LPLP generalizes local privacy by flexibly specifyingpairs of observations to be rendered indistinguishable. Insection 3, we describe the geometric mechanism of Ghoshet al. (2012) and prove that it is a mechanism for LPLP.

Differentially private Bayesian inference. In Bayesianstatistics, we begin with a probabilistic model M thatrelates observable variables Y to latent variables Z via ajoint distribution PM(Y, Z). The goal of inference is thento compute the posterior distribution PM(Z |Y ) over thelatent variables conditioned on observed values of Y . Theposterior is almost always analytically intractable and thusinference involves approximating it. The two most commonmethods of approximate Bayesian inference are variationalinference, wherein we fit the parameters of an approximat-ing distribution Q(Z |Y ), and Markov chain Monte Carlo(MCMC), wherein we approximate the posterior with a

finite set of samples {Z(s)}Ss=1 generated via a Markovchain whose stationary distribution is the exact posterior.

We can conceptualize Bayesian inference as a randomizedalgorithm A(·) that returns an approximation to theposterior distribution PM(Z |Y ). In general A(·) doesnot satisfy ε-differential privacy. However, if A(·) isan MCMC algorithm that returns a single sample fromthe posterior, it guarantees privacy (Dimitrakakis et al.,2014; Wang et al., 2015; Foulds et al., 2016; Dimitrakakiset al., 2017). Adding noise to posterior samples can alsoguarantee privacy (Zhang et al., 2016), though this set ofnoised samples {Z(s)}Ss=1 approximates some distributionPM(Z |Y ) that depends on ε and is different than the exactposterior (but close, in some sense, and equal when ε→∞).For specific models, we can also noise the transition kernelof the MCMC algorithm to construct a Markov chain whosestationary distribution is again not the exact posterior, butsomething close that guarantees privacy (Foulds et al.,2016). We can also take an analogous approach to privatizevariational inference, wherein we add noise to the sufficientstatistics computed in each iteration (Park et al., 2016).

Locally private Bayesian inference. We first formalize thegeneral objective of Bayesian inference under local privacy.Given a generative model M for non-privatized data setY and latent variables Z with joint distribution PM(Y,Z),we further assume a randomized response methodR(·) thatgenerates privatized data sets: Y ∼ PR(Y |Y ). The infer-ence goal is then to approximate the locally private posterior

PM,R(Z | Y ) = EPM,R(Y | Y ) [PM(Z |Y )]

=

∫PM(Z |Y )PM,R(Y | Y ) dY. (4)

This distribution correctly characterizes our uncertaintyabout the latent variables Z, conditioned on all of ourobservations and assumptions—i.e., the privatized dataset Y , the model M, and the randomized responsemethod R. The expansion in equation 4 shows that thisposterior implicitly treats the non-privatized data set Y asa latent variable and marginalizes over it using the mixingdistribution PM,R(Y | Y ) which is itself a posterior thatcharacterizes our uncertainty about Y . The key observationhere is that if we can generate samples from PM,R(Y | Y ),we can use them to approximate the expectation inequation 4, assuming that we already have a method forapproximating the non-private posterior PM(Z |Y ). In thecontext of MCMC, iteratively re-sampling values of thenon-privatized data set from its complete conditional—i.e.,Y (s) ∼ PM,R(Y |Z(s−1), Y )—and then re-samplingvalues of the latent variables—i.e., Z(s) ∼ PM(Z |Y (s))—constitutes a Markov chain whose stationary distributionis PM,R(Z, Y | Y ). In scenarios where we already havederivations and implementations for sampling fromPM(Z |Y ), we need only be able to sample efficiently


from PM,R(Y |Z, Y ) in order to obtain a locally privateBayesian inference algorithm; whether we can do thisefficiently depends heavily on our choice ofM andR.

We note that the objective of Bayesian inference underlocal privacy, as defined in equation 4, is similar to thatof Williams & McSherry (2010), who identify their keybarrier to inference as being unable to analytically form themarginal likelihood that links the privatized data set to Z:

PM,R(Y |Z) =

∫PR(Y |Y )PM(Y |Z) dY. (5)

In the next sections, we show that ifM is a Poisson factor-ization model andR is the geometric mechanism, then wecan analytically form an augmented version of this marginallikelihood and derive an MCMC algorithm that samplesefficiently from the locally private posterior in equation 4.

3. Locally private Poisson factorizationIn this section, we describe a model M—i.e., Poissonfactorization—and a randomized response methodR—i.e.,the geometric mechanism—each of which is a natural choicefor count data. We prove two theorems about the geometricmechanism: 1) it is a mechanism for LPLP and 2) it can bere-interpreted in terms of the Skellam distribution (Skellam,1946). We rely on the second theorem to show that Poissonfactorization and the geometric mechanism combine to yielda novel generative process for privatized count data, whichwe then exploit in section 4 to derive our MCMC algorithm.

M: Poisson factorization. We assume that Y is a dataset that consists of counts, each of which yi ∈ Z+ is anindependent Poisson random variable yi∼Pois(µi) wherethe rate parameter µi is defined to be a deterministicfunction of the latent variables Z. The subscript i is amulti-index. In Poisson matrix factorization, i ≡ (i1, i2);however, this notation also supports Poisson tensorfactorization, where i ≡ (i1, . . . , iM ), and multiviewmodels, where the multi-index may differ in the numberof indices. This class of models includes some of the mostwidely used models in the social sciences, as described insection 1. In section 5, we present case studies involvingtwo different models—specifically, the mixed-membershipstochastic block model for social networks (Ball et al.,2011; Gopalan & Blei, 2013; Zhou, 2015) and latentDirichlet allocation (Blei et al., 2003). Although both ofthese models are instances of Poisson matrix factorization,our method applies to any Poisson factorization model.

R: Geometric mechanism. The two most commonly usedrandomized response methods in the differential privacytoolbox—the Gaussian and Laplace mechanisms—privatizeobservations by adding noise drawn from real-valued distri-butions. They are therefore unnatural choices for count data.Ghosh et al. (2012) introduced the geometric mechanism,

which can be viewed as the discrete analog of the Laplacemechanism and involves adding integer-valued noise τ ∈ Zdrawn from the two-sided geometric distribution. The PMFfor the two-sided geometric distribution is as follows:

2Geo(τ ;α) =1− α1 + α

α|τ |. (6)

Theorem 1. (Proof in appendix) Let randomized responsemethodR(·) be the geometric mechanism with parameterα. Then for any positive integer N , and any pair of obser-vations y, y′ ∈ Y such that ‖y − y′‖1 ≤ N ,R(·) satisfies

P (R(y) ∈ S) ≤ eεP (R(y′) ∈ S) (7)

for all subsets S in the range ofR(·), where

ε = N ln( 1

α

). (8)

Therefore, for any positive integer N , the geometric mech-anism with parameter α is an (N, ε)-private randomized re-sponse method with ε = N ln ( 1

α ). If ε′

N ′ = εN , then the geo-

metric mechanism with parameter α is also (N ′, ε′)-private.Theorem 2. (Proof in appendix) A two-sided geometricrandom variable τ ∼ 2Geo(α) can be generated as follows:

τ ∼ Skel(λ(+), λ(−)), λ(∗) ∼ Exp( α1−α ), (9)

where Exp(·) and Skel(·) are the exponential and Skellamdistributions. The latter is the marginal distribution of thedifference τ := g(+)−g(−) of two independent Poissonrandom variables g(∗) ∼ Pois(λ(∗)), where ∗ ∈ {+,−}.

CombiningM andR. We assume that each non-privatizedcount yi is generated byM and then privatized byR:

y(±)

i := yi + τi, τi ∼ 2Geo(α), (10)

where y(±)

i is the privatized count, which we superscriptwith (±) to denote that it may be non-negative or negativebecause the additive noise τi∈Z may itself be negative.

Via theorem 2, we can express the generative process fory(±)

i in four equivalent ways, shown in figure 2, each ofwhich provides a unique and necessary insight. Process 1 isa graphical representation of the generative process as de-scribed above. Process 2 represents the two-sided geometricnoise in terms of a pair of Poisson random variables withexponentially distributed rates; in so doing, it reveals theauxiliary variables that facilitate inference. Process 3 repre-sents the sum of the non-privatized count and the positivecomponent of the noise as a single Poisson random variabley(+)

i =yi + g(+)

i . Process 4 marginalizes out the remainingPoisson random variables to yield a representation of y(±)

i

as an exponentially randomized Skellam random variable:

y(±)

i ∼Skel(λ(+)

i +µi, λ(−)

i

), λ(∗)

i ∼Exp( α1−α ). (11)

The derivations of generative processes 2, 3, and 4 rely onproperties of the two-sided geometric, Skellam, and Poissondistributions. We provide these properties in the appendix.


y<latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit>

↵<latexit sha1_base64="JtDqaCSYHdUsArJlViGZOYtHm8o=">AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzAOSJfROJsmY2ZllZlYIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSgQ31ve/vcLa+sbmVnG7tLO7t39QPjxqGpVqyhpUCaXbERomuGQNy61g7UQzjCPBWtH4dua3npg2XMkHO0lYGONQ8gGnaJ3U7KJIRtgrV/yqPwdZJUFOKpCj3it/dfuKpjGTlgo0phP4iQ0z1JZTwaalbmpYgnSMQ9ZxVGLMTJjNr52SM6f0yUBpV9KSufp7IsPYmEkcuc4Y7cgsezPxP6+T2sF1mHGZpJZJulg0SAWxisxeJ32uGbVi4ghSzd2thI5QI7UuoJILIVh+eZU0L6qBXw3uLyu1mzyOIpzAKZxDAFdQgzuoQwMoPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBi4GPGA==</latexit><latexit sha1_base64="JtDqaCSYHdUsArJlViGZOYtHm8o=">AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzAOSJfROJsmY2ZllZlYIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSgQ31ve/vcLa+sbmVnG7tLO7t39QPjxqGpVqyhpUCaXbERomuGQNy61g7UQzjCPBWtH4dua3npg2XMkHO0lYGONQ8gGnaJ3U7KJIRtgrV/yqPwdZJUFOKpCj3it/dfuKpjGTlgo0phP4iQ0z1JZTwaalbmpYgnSMQ9ZxVGLMTJjNr52SM6f0yUBpV9KSufp7IsPYmEkcuc4Y7cgsezPxP6+T2sF1mHGZpJZJulg0SAWxisxeJ32uGbVi4ghSzd2thI5QI7UuoJILIVh+eZU0L6qBXw3uLyu1mzyOIpzAKZxDAFdQgzuoQwMoPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBi4GPGA==</latexit><latexit sha1_base64="JtDqaCSYHdUsArJlViGZOYtHm8o=">AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzAOSJfROJsmY2ZllZlYIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSgQ31ve/vcLa+sbmVnG7tLO7t39QPjxqGpVqyhpUCaXbERomuGQNy61g7UQzjCPBWtH4dua3npg2XMkHO0lYGONQ8gGnaJ3U7KJIRtgrV/yqPwdZJUFOKpCj3it/dfuKpjGTlgo0phP4iQ0z1JZTwaalbmpYgnSMQ9ZxVGLMTJjNr52SM6f0yUBpV9KSufp7IsPYmEkcuc4Y7cgsezPxP6+T2sF1mHGZpJZJulg0SAWxisxeJ32uGbVi4ghSzd2thI5QI7UuoJILIVh+eZU0L6qBXw3uLyu1mzyOIpzAKZxDAFdQgzuoQwMoPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBi4GPGA==</latexit><latexit sha1_base64="JtDqaCSYHdUsArJlViGZOYtHm8o=">AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzAOSJfROJsmY2ZllZlYIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSgQ31ve/vcLa+sbmVnG7tLO7t39QPjxqGpVqyhpUCaXbERomuGQNy61g7UQzjCPBWtH4dua3npg2XMkHO0lYGONQ8gGnaJ3U7KJIRtgrV/yqPwdZJUFOKpCj3it/dfuKpjGTlgo0phP4iQ0z1JZTwaalbmpYgnSMQ9ZxVGLMTJjNr52SM6f0yUBpV9KSufp7IsPYmEkcuc4Y7cgsezPxP6+T2sF1mHGZpJZJulg0SAWxisxeJ32uGbVi4ghSzd2thI5QI7UuoJILIVh+eZU0L6qBXw3uLyu1mzyOIpzAKZxDAFdQgzuoQwMoPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBi4GPGA==</latexit>

µ<latexit sha1_base64="UFOX4zita877+Ikq+M6IENXmVh0=">AAAB6nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzQOSJcxOZpMhM7PLTK8QQj7BiwdFvPpF3vwbJ8keNLGgoajqprsrSqWw6PvfXmFtfWNzq7hd2tnd2z8oHx41bZIZxhsskYlpR9RyKTRvoEDJ26nhVEWSt6LR7cxvPXFjRaIfcZzyUNGBFrFgFJ300FVZr1zxq/4cZJUEOalAjnqv/NXtJyxTXCOT1NpO4KcYTqhBwSSflrqZ5SllIzrgHUc1VdyGk/mpU3LmlD6JE+NKI5mrvycmVFk7VpHrVBSHdtmbif95nQzj63AidJoh12yxKM4kwYTM/iZ9YThDOXaEMiPcrYQNqaEMXTolF0Kw/PIqaV5UA78a3F9Wajd5HEU4gVM4hwCuoAZ3UIcGMBjAM7zCmye9F+/d+1i0Frx85hj+wPv8AV1ejdY=</latexit><latexit sha1_base64="UFOX4zita877+Ikq+M6IENXmVh0=">AAAB6nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzQOSJcxOZpMhM7PLTK8QQj7BiwdFvPpF3vwbJ8keNLGgoajqprsrSqWw6PvfXmFtfWNzq7hd2tnd2z8oHx41bZIZxhsskYlpR9RyKTRvoEDJ26nhVEWSt6LR7cxvPXFjRaIfcZzyUNGBFrFgFJ300FVZr1zxq/4cZJUEOalAjnqv/NXtJyxTXCOT1NpO4KcYTqhBwSSflrqZ5SllIzrgHUc1VdyGk/mpU3LmlD6JE+NKI5mrvycmVFk7VpHrVBSHdtmbif95nQzj63AidJoh12yxKM4kwYTM/iZ9YThDOXaEMiPcrYQNqaEMXTolF0Kw/PIqaV5UA78a3F9Wajd5HEU4gVM4hwCuoAZ3UIcGMBjAM7zCmye9F+/d+1i0Frx85hj+wPv8AV1ejdY=</latexit><latexit sha1_base64="UFOX4zita877+Ikq+M6IENXmVh0=">AAAB6nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzQOSJcxOZpMhM7PLTK8QQj7BiwdFvPpF3vwbJ8keNLGgoajqprsrSqWw6PvfXmFtfWNzq7hd2tnd2z8oHx41bZIZxhsskYlpR9RyKTRvoEDJ26nhVEWSt6LR7cxvPXFjRaIfcZzyUNGBFrFgFJ300FVZr1zxq/4cZJUEOalAjnqv/NXtJyxTXCOT1NpO4KcYTqhBwSSflrqZ5SllIzrgHUc1VdyGk/mpU3LmlD6JE+NKI5mrvycmVFk7VpHrVBSHdtmbif95nQzj63AidJoh12yxKM4kwYTM/iZ9YThDOXaEMiPcrYQNqaEMXTolF0Kw/PIqaV5UA78a3F9Wajd5HEU4gVM4hwCuoAZ3UIcGMBjAM7zCmye9F+/d+1i0Frx85hj+wPv8AV1ejdY=</latexit><latexit sha1_base64="UFOX4zita877+Ikq+M6IENXmVh0=">AAAB6nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzQOSJcxOZpMhM7PLTK8QQj7BiwdFvPpF3vwbJ8keNLGgoajqprsrSqWw6PvfXmFtfWNzq7hd2tnd2z8oHx41bZIZxhsskYlpR9RyKTRvoEDJ26nhVEWSt6LR7cxvPXFjRaIfcZzyUNGBFrFgFJ300FVZr1zxq/4cZJUEOalAjnqv/NXtJyxTXCOT1NpO4KcYTqhBwSSflrqZ5SllIzrgHUc1VdyGk/mpU3LmlD6JE+NKI5mrvycmVFk7VpHrVBSHdtmbif95nQzj63AidJoh12yxKM4kwYTM/iZ9YThDOXaEMiPcrYQNqaEMXTolF0Kw/PIqaV5UA78a3F9Wajd5HEU4gVM4hwCuoAZ3UIcGMBjAM7zCmye9F+/d+1i0Frx85hj+wPv8AV1ejdY=</latexit>

y(±)

<latexit sha1_base64="PReL3nbb5Njp3rP+7Xilw1f/zP0=">AAACJHicbVDLSsNAFJ34rPVVdenCYBHqpiQi6EoKblxWsA9oaplMb9qhM5MwMxFiyNJfceNW/8KduHDjJ/gNTtMutO2BC4dz7oVzjx8xqrTjfFlLyyura+uFjeLm1vbObmlvv6nCWBJokJCFsu1jBYwKaGiqGbQjCZj7DFr+6Hrstx5AKhqKO51E0OV4IGhACdZG6pWOPE1ZH9Iku089jvVQccwYyLTiRfw0y3qlslN1ctjzxJ2SMpqi3iv9eP2QxByEJgwr1XGdSHdTLDUlDLKiFyuIMBnhAXQMFZiD6qb5I5l9YpS+HYTSjNB2rv69SDFXKuG+2cyzznpjcaEngSn6CIu8TqyDy25KRRRrEGQSIoiZrUN73JjdpxKIZokhmEhq/rDJEEtMtOm1aApyZ+uYJ82zqutU3dvzcu1qWlUBHaJjVEEuukA1dIPqqIEIekIv6BW9Wc/Wu/VhfU5Wl6zpzQH6B+v7FyBZpgk=</latexit><latexit sha1_base64="PReL3nbb5Njp3rP+7Xilw1f/zP0=">AAACJHicbVDLSsNAFJ34rPVVdenCYBHqpiQi6EoKblxWsA9oaplMb9qhM5MwMxFiyNJfceNW/8KduHDjJ/gNTtMutO2BC4dz7oVzjx8xqrTjfFlLyyura+uFjeLm1vbObmlvv6nCWBJokJCFsu1jBYwKaGiqGbQjCZj7DFr+6Hrstx5AKhqKO51E0OV4IGhACdZG6pWOPE1ZH9Iku089jvVQccwYyLTiRfw0y3qlslN1ctjzxJ2SMpqi3iv9eP2QxByEJgwr1XGdSHdTLDUlDLKiFyuIMBnhAXQMFZiD6qb5I5l9YpS+HYTSjNB2rv69SDFXKuG+2cyzznpjcaEngSn6CIu8TqyDy25KRRRrEGQSIoiZrUN73JjdpxKIZokhmEhq/rDJEEtMtOm1aApyZ+uYJ82zqutU3dvzcu1qWlUBHaJjVEEuukA1dIPqqIEIekIv6BW9Wc/Wu/VhfU5Wl6zpzQH6B+v7FyBZpgk=</latexit><latexit sha1_base64="PReL3nbb5Njp3rP+7Xilw1f/zP0=">AAACJHicbVDLSsNAFJ34rPVVdenCYBHqpiQi6EoKblxWsA9oaplMb9qhM5MwMxFiyNJfceNW/8KduHDjJ/gNTtMutO2BC4dz7oVzjx8xqrTjfFlLyyura+uFjeLm1vbObmlvv6nCWBJokJCFsu1jBYwKaGiqGbQjCZj7DFr+6Hrstx5AKhqKO51E0OV4IGhACdZG6pWOPE1ZH9Iku089jvVQccwYyLTiRfw0y3qlslN1ctjzxJ2SMpqi3iv9eP2QxByEJgwr1XGdSHdTLDUlDLKiFyuIMBnhAXQMFZiD6qb5I5l9YpS+HYTSjNB2rv69SDFXKuG+2cyzznpjcaEngSn6CIu8TqyDy25KRRRrEGQSIoiZrUN73JjdpxKIZokhmEhq/rDJEEtMtOm1aApyZ+uYJ82zqutU3dvzcu1qWlUBHaJjVEEuukA1dIPqqIEIekIv6BW9Wc/Wu/VhfU5Wl6zpzQH6B+v7FyBZpgk=</latexit><latexit sha1_base64="PReL3nbb5Njp3rP+7Xilw1f/zP0=">AAACJHicbVDLSsNAFJ34rPVVdenCYBHqpiQi6EoKblxWsA9oaplMb9qhM5MwMxFiyNJfceNW/8KduHDjJ/gNTtMutO2BC4dz7oVzjx8xqrTjfFlLyyura+uFjeLm1vbObmlvv6nCWBJokJCFsu1jBYwKaGiqGbQjCZj7DFr+6Hrstx5AKhqKO51E0OV4IGhACdZG6pWOPE1ZH9Iku089jvVQccwYyLTiRfw0y3qlslN1ctjzxJ2SMpqi3iv9eP2QxByEJgwr1XGdSHdTLDUlDLKiFyuIMBnhAXQMFZiD6qb5I5l9YpS+HYTSjNB2rv69SDFXKuG+2cyzznpjcaEngSn6CIu8TqyDy25KRRRrEGQSIoiZrUN73JjdpxKIZokhmEhq/rDJEEtMtOm1aApyZ+uYJ82zqutU3dvzcu1qWlUBHaJjVEEuukA1dIPqqIEIekIv6BW9Wc/Wu/VhfU5Wl6zpzQH6B+v7FyBZpgk=</latexit>

⌧<latexit sha1_base64="rx/s5aCvXUP35GpplwAkBrTmD4o=">AAAB63icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0DJoYxnBxEByhL3NJlmyu3fszgnhyF+wsVDE1j9k579xL7lCEx8MPN6bYWZelEhh0fe/vdLa+sbmVnm7srO7t39QPTxq2zg1jLdYLGPTiajlUmjeQoGSdxLDqYokf4wmt7n/+MSNFbF+wGnCQ0VHWgwFo5hLPaRpv1rz6/4cZJUEBalBgWa/+tUbxCxVXCOT1Npu4CcYZtSgYJLPKr3U8oSyCR3xrqOaKm7DbH7rjJw5ZUCGsXGlkczV3xMZVdZOVeQ6FcWxXfZy8T+vm+LwOsyETlLkmi0WDVNJMCb542QgDGcop45QZoS7lbAxNZShi6fiQgiWX14l7Yt64NeD+8ta46aIowwncArnEMAVNOAOmtACBmN4hld485T34r17H4vWklfMHMMfeJ8/IS2OSA==</latexit><latexit sha1_base64="rx/s5aCvXUP35GpplwAkBrTmD4o=">AAAB63icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0DJoYxnBxEByhL3NJlmyu3fszgnhyF+wsVDE1j9k579xL7lCEx8MPN6bYWZelEhh0fe/vdLa+sbmVnm7srO7t39QPTxq2zg1jLdYLGPTiajlUmjeQoGSdxLDqYokf4wmt7n/+MSNFbF+wGnCQ0VHWgwFo5hLPaRpv1rz6/4cZJUEBalBgWa/+tUbxCxVXCOT1Npu4CcYZtSgYJLPKr3U8oSyCR3xrqOaKm7DbH7rjJw5ZUCGsXGlkczV3xMZVdZOVeQ6FcWxXfZy8T+vm+LwOsyETlLkmi0WDVNJMCb542QgDGcop45QZoS7lbAxNZShi6fiQgiWX14l7Yt64NeD+8ta46aIowwncArnEMAVNOAOmtACBmN4hld485T34r17H4vWklfMHMMfeJ8/IS2OSA==</latexit><latexit sha1_base64="rx/s5aCvXUP35GpplwAkBrTmD4o=">AAAB63icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0DJoYxnBxEByhL3NJlmyu3fszgnhyF+wsVDE1j9k579xL7lCEx8MPN6bYWZelEhh0fe/vdLa+sbmVnm7srO7t39QPTxq2zg1jLdYLGPTiajlUmjeQoGSdxLDqYokf4wmt7n/+MSNFbF+wGnCQ0VHWgwFo5hLPaRpv1rz6/4cZJUEBalBgWa/+tUbxCxVXCOT1Npu4CcYZtSgYJLPKr3U8oSyCR3xrqOaKm7DbH7rjJw5ZUCGsXGlkczV3xMZVdZOVeQ6FcWxXfZy8T+vm+LwOsyETlLkmi0WDVNJMCb542QgDGcop45QZoS7lbAxNZShi6fiQgiWX14l7Yt64NeD+8ta46aIowwncArnEMAVNOAOmtACBmN4hld485T34r17H4vWklfMHMMfeJ8/IS2OSA==</latexit><latexit sha1_base64="rx/s5aCvXUP35GpplwAkBrTmD4o=">AAAB63icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0DJoYxnBxEByhL3NJlmyu3fszgnhyF+wsVDE1j9k579xL7lCEx8MPN6bYWZelEhh0fe/vdLa+sbmVnm7srO7t39QPTxq2zg1jLdYLGPTiajlUmjeQoGSdxLDqYokf4wmt7n/+MSNFbF+wGnCQ0VHWgwFo5hLPaRpv1rz6/4cZJUEBalBgWa/+tUbxCxVXCOT1Npu4CcYZtSgYJLPKr3U8oSyCR3xrqOaKm7DbH7rjJw5ZUCGsXGlkczV3xMZVdZOVeQ6FcWxXfZy8T+vm+LwOsyETlLkmi0WDVNJMCb542QgDGcop45QZoS7lbAxNZShi6fiQgiWX14l7Yt64NeD+8ta46aIowwncArnEMAVNOAOmtACBmN4hld485T34r17H4vWklfMHMMfeJ8/IS2OSA==</latexit>

y<latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit><latexit sha1_base64="l29WxoUb9DEbvmhLG7jHtZ0OU24=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68diC/YA2lM120q7dbMLuRgihv8CLB0W8+pO8+W/ctjlo64OBx3szzMwLEsG1cd1vZ219Y3Nru7RT3t3bPzisHB23dZwqhi0Wi1h1A6pRcIktw43AbqKQRoHATjC5m/mdJ1Sax/LBZAn6ER1JHnJGjZWa2aBSdWvuHGSVeAWpQoHGoPLVH8YsjVAaJqjWPc9NjJ9TZTgTOC33U40JZRM6wp6lkkao/Xx+6JScW2VIwljZkobM1d8TOY20zqLAdkbUjPWyNxP/83qpCW/8nMskNSjZYlGYCmJiMvuaDLlCZkRmCWWK21sJG1NFmbHZlG0I3vLLq6R9WfPcmte8qtZvizhKcApncAEeXEMd7qEBLWCA8Ayv8OY8Oi/Ou/OxaF1zipkT+APn8wfnvYz9</latexit>



y(±)


⌧<latexit sha1_base64="rx/s5aCvXUP35GpplwAkBrTmD4o=">AAAB63icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0DJoYxnBxEByhL3NJlmyu3fszgnhyF+wsVDE1j9k579xL7lCEx8MPN6bYWZelEhh0fe/vdLa+sbmVnm7srO7t39QPTxq2zg1jLdYLGPTiajlUmjeQoGSdxLDqYokf4wmt7n/+MSNFbF+wGnCQ0VHWgwFo5hLPaRpv1rz6/4cZJUEBalBgWa/+tUbxCxVXCOT1Npu4CcYZtSgYJLPKr3U8oSyCR3xrqOaKm7DbH7rjJw5ZUCGsXGlkczV3xMZVdZOVeQ6FcWxXfZy8T+vm+LwOsyETlLkmi0WDVNJMCb542QgDGcop45QZoS7lbAxNZShi6fiQgiWX14l7Yt64NeD+8ta46aIowwncArnEMAVNOAOmtACBmN4hld485T34r17H4vWklfMHMMfeJ8/IS2OSA==</latexit><latexit sha1_base64="rx/s5aCvXUP35GpplwAkBrTmD4o=">AAAB63icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0DJoYxnBxEByhL3NJlmyu3fszgnhyF+wsVDE1j9k579xL7lCEx8MPN6bYWZelEhh0fe/vdLa+sbmVnm7srO7t39QPTxq2zg1jLdYLGPTiajlUmjeQoGSdxLDqYokf4wmt7n/+MSNFbF+wGnCQ0VHWgwFo5hLPaRpv1rz6/4cZJUEBalBgWa/+tUbxCxVXCOT1Npu4CcYZtSgYJLPKr3U8oSyCR3xrqOaKm7DbH7rjJw5ZUCGsXGlkczV3xMZVdZOVeQ6FcWxXfZy8T+vm+LwOsyETlLkmi0WDVNJMCb542QgDGcop45QZoS7lbAxNZShi6fiQgiWX14l7Yt64NeD+8ta46aIowwncArnEMAVNOAOmtACBmN4hld485T34r17H4vWklfMHMMfeJ8/IS2OSA==</latexit><latexit sha1_base64="rx/s5aCvXUP35GpplwAkBrTmD4o=">AAAB63icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0DJoYxnBxEByhL3NJlmyu3fszgnhyF+wsVDE1j9k579xL7lCEx8MPN6bYWZelEhh0fe/vdLa+sbmVnm7srO7t39QPTxq2zg1jLdYLGPTiajlUmjeQoGSdxLDqYokf4wmt7n/+MSNFbF+wGnCQ0VHWgwFo5hLPaRpv1rz6/4cZJUEBalBgWa/+tUbxCxVXCOT1Npu4CcYZtSgYJLPKr3U8oSyCR3xrqOaKm7DbH7rjJw5ZUCGsXGlkczV3xMZVdZOVeQ6FcWxXfZy8T+vm+LwOsyETlLkmi0WDVNJMCb542QgDGcop45QZoS7lbAxNZShi6fiQgiWX14l7Yt64NeD+8ta46aIowwncArnEMAVNOAOmtACBmN4hld485T34r17H4vWklfMHMMfeJ8/IS2OSA==</latexit><latexit sha1_base64="rx/s5aCvXUP35GpplwAkBrTmD4o=">AAAB63icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0DJoYxnBxEByhL3NJlmyu3fszgnhyF+wsVDE1j9k579xL7lCEx8MPN6bYWZelEhh0fe/vdLa+sbmVnm7srO7t39QPTxq2zg1jLdYLGPTiajlUmjeQoGSdxLDqYokf4wmt7n/+MSNFbF+wGnCQ0VHWgwFo5hLPaRpv1rz6/4cZJUEBalBgWa/+tUbxCxVXCOT1Npu4CcYZtSgYJLPKr3U8oSyCR3xrqOaKm7DbH7rjJw5ZUCGsXGlkczV3xMZVdZOVeQ6FcWxXfZy8T+vm+LwOsyETlLkmi0WDVNJMCb542QgDGcop45QZoS7lbAxNZShi6fiQgiWX14l7Yt64NeD+8ta46aIowwncArnEMAVNOAOmtACBmN4hld485T34r17H4vWklfMHMMfeJ8/IS2OSA==</latexit>

g(�)

<latexit sha1_base64="jVLRleK787/Z2+P+899rA32Gnew=">AAACGnicbVDLSsNAFJ3UV62vqLhyEyxCXVgSEXQlBTcuK9gHtLFMprft0JlJmJkINeRP3LjVv3Anbt34E36D0zQLrT1w4XDOvXDuCSJGlXbdL6uwtLyyulZcL21sbm3v2Lt7TRXGkkCDhCyU7QArYFRAQ1PNoB1JwDxg0ArG11O/9QBS0VDc6UkEPsdDQQeUYG2knn0wvE+6HOuR4pgxkEnl9CRNe3bZrboZnP/Ey0kZ5aj37O9uPyQxB6EJw0p1PDfSfoKlpoRBWurGCiJMxngIHUMF5qD8JIufOsdG6TuDUJoR2snU3xcJ5kpNeGA2s6Tz3lRc6Elgij7CIq8T68Gln1ARxRoEmYUYxMzRoTPtyelTCUSziSGYSGr+cMgIS0y0abNkCvLm6/hPmmdVz616t+fl2lVeVREdoiNUQR66QDV0g+qogQhK0DN6Qa/Wk/VmvVsfs9WCld/soz+wPn8A5wmhIQ==</latexit><latexit sha1_base64="jVLRleK787/Z2+P+899rA32Gnew=">AAACGnicbVDLSsNAFJ3UV62vqLhyEyxCXVgSEXQlBTcuK9gHtLFMprft0JlJmJkINeRP3LjVv3Anbt34E36D0zQLrT1w4XDOvXDuCSJGlXbdL6uwtLyyulZcL21sbm3v2Lt7TRXGkkCDhCyU7QArYFRAQ1PNoB1JwDxg0ArG11O/9QBS0VDc6UkEPsdDQQeUYG2knn0wvE+6HOuR4pgxkEnl9CRNe3bZrboZnP/Ey0kZ5aj37O9uPyQxB6EJw0p1PDfSfoKlpoRBWurGCiJMxngIHUMF5qD8JIufOsdG6TuDUJoR2snU3xcJ5kpNeGA2s6Tz3lRc6Elgij7CIq8T68Gln1ARxRoEmYUYxMzRoTPtyelTCUSziSGYSGr+cMgIS0y0abNkCvLm6/hPmmdVz616t+fl2lVeVREdoiNUQR66QDV0g+qogQhK0DN6Qa/Wk/VmvVsfs9WCld/soz+wPn8A5wmhIQ==</latexit><latexit sha1_base64="jVLRleK787/Z2+P+899rA32Gnew=">AAACGnicbVDLSsNAFJ3UV62vqLhyEyxCXVgSEXQlBTcuK9gHtLFMprft0JlJmJkINeRP3LjVv3Anbt34E36D0zQLrT1w4XDOvXDuCSJGlXbdL6uwtLyyulZcL21sbm3v2Lt7TRXGkkCDhCyU7QArYFRAQ1PNoB1JwDxg0ArG11O/9QBS0VDc6UkEPsdDQQeUYG2knn0wvE+6HOuR4pgxkEnl9CRNe3bZrboZnP/Ey0kZ5aj37O9uPyQxB6EJw0p1PDfSfoKlpoRBWurGCiJMxngIHUMF5qD8JIufOsdG6TuDUJoR2snU3xcJ5kpNeGA2s6Tz3lRc6Elgij7CIq8T68Gln1ARxRoEmYUYxMzRoTPtyelTCUSziSGYSGr+cMgIS0y0abNkCvLm6/hPmmdVz616t+fl2lVeVREdoiNUQR66QDV0g+qogQhK0DN6Qa/Wk/VmvVsfs9WCld/soz+wPn8A5wmhIQ==</latexit><latexit sha1_base64="jVLRleK787/Z2+P+899rA32Gnew=">AAACGnicbVDLSsNAFJ3UV62vqLhyEyxCXVgSEXQlBTcuK9gHtLFMprft0JlJmJkINeRP3LjVv3Anbt34E36D0zQLrT1w4XDOvXDuCSJGlXbdL6uwtLyyulZcL21sbm3v2Lt7TRXGkkCDhCyU7QArYFRAQ1PNoB1JwDxg0ArG11O/9QBS0VDc6UkEPsdDQQeUYG2knn0wvE+6HOuR4pgxkEnl9CRNe3bZrboZnP/Ey0kZ5aj37O9uPyQxB6EJw0p1PDfSfoKlpoRBWurGCiJMxngIHUMF5qD8JIufOsdG6TuDUJoR2snU3xcJ5kpNeGA2s6Tz3lRc6Elgij7CIq8T68Gln1ARxRoEmYUYxMzRoTPtyelTCUSziSGYSGr+cMgIS0y0abNkCvLm6/hPmmdVz616t+fl2lVeVREdoiNUQR66QDV0g+qogQhK0DN6Qa/Wk/VmvVsfs9WCld/soz+wPn8A5wmhIQ==</latexit>

�(+)<latexit sha1_base64="K/hyhHL94xYxP88/ls/AtH52iBc=">AAACGXicbVDLSsNAFJ34rPUVddlNsAgVoSQi6EoKblxWsA9oYplMbtqhk0mYmQg1dOF3+AFu9RPciVtXfoG/4TTNQlsPDBzOOZc79/gJo1LZ9pextLyyurZe2ihvbm3v7Jp7+20Zp4JAi8QsFl0fS2CUQ0tRxaCbCMCRz6Djj66mfucehKQxv1XjBLwIDzgNKcFKS32z4jIdDvBd5kZYDWWEGQOR1U6OJ5O+WbXrdg5rkTgFqaICzb757QYxSSPgijAsZc+xE+VlWChKGEzKbiohwWSEB9DTlOMIpJflR0ysI60EVhgL/biycvX3RIYjKceRr5P5T+e9qfivJ4BJ+jC/XoUXXkZ5kirgZLY9TJmlYmtakxVQAUSxsSaYCKoPsMgQC0yULrOsm3Hme1gk7dO6Y9edm7Nq47LoqIQq6BDVkIPOUQNdoyZqIYIe0TN6Qa/Gk/FmvBsfs+iSUcwcoD8wPn8AsK2hJQ==</latexit><latexit sha1_base64="K/hyhHL94xYxP88/ls/AtH52iBc=">AAACGXicbVDLSsNAFJ34rPUVddlNsAgVoSQi6EoKblxWsA9oYplMbtqhk0mYmQg1dOF3+AFu9RPciVtXfoG/4TTNQlsPDBzOOZc79/gJo1LZ9pextLyyurZe2ihvbm3v7Jp7+20Zp4JAi8QsFl0fS2CUQ0tRxaCbCMCRz6Djj66mfucehKQxv1XjBLwIDzgNKcFKS32z4jIdDvBd5kZYDWWEGQOR1U6OJ5O+WbXrdg5rkTgFqaICzb757QYxSSPgijAsZc+xE+VlWChKGEzKbiohwWSEB9DTlOMIpJflR0ysI60EVhgL/biycvX3RIYjKceRr5P5T+e9qfivJ4BJ+jC/XoUXXkZ5kirgZLY9TJmlYmtakxVQAUSxsSaYCKoPsMgQC0yULrOsm3Hme1gk7dO6Y9edm7Nq47LoqIQq6BDVkIPOUQNdoyZqIYIe0TN6Qa/Gk/FmvBsfs+iSUcwcoD8wPn8AsK2hJQ==</latexit><latexit sha1_base64="K/hyhHL94xYxP88/ls/AtH52iBc=">AAACGXicbVDLSsNAFJ34rPUVddlNsAgVoSQi6EoKblxWsA9oYplMbtqhk0mYmQg1dOF3+AFu9RPciVtXfoG/4TTNQlsPDBzOOZc79/gJo1LZ9pextLyyurZe2ihvbm3v7Jp7+20Zp4JAi8QsFl0fS2CUQ0tRxaCbCMCRz6Djj66mfucehKQxv1XjBLwIDzgNKcFKS32z4jIdDvBd5kZYDWWEGQOR1U6OJ5O+WbXrdg5rkTgFqaICzb757QYxSSPgijAsZc+xE+VlWChKGEzKbiohwWSEB9DTlOMIpJflR0ysI60EVhgL/biycvX3RIYjKceRr5P5T+e9qfivJ4BJ+jC/XoUXXkZ5kirgZLY9TJmlYmtakxVQAUSxsSaYCKoPsMgQC0yULrOsm3Hme1gk7dO6Y9edm7Nq47LoqIQq6BDVkIPOUQNdoyZqIYIe0TN6Qa/Gk/FmvBsfs+iSUcwcoD8wPn8AsK2hJQ==</latexit><latexit sha1_base64="K/hyhHL94xYxP88/ls/AtH52iBc=">AAACGXicbVDLSsNAFJ34rPUVddlNsAgVoSQi6EoKblxWsA9oYplMbtqhk0mYmQg1dOF3+AFu9RPciVtXfoG/4TTNQlsPDBzOOZc79/gJo1LZ9pextLyyurZe2ihvbm3v7Jp7+20Zp4JAi8QsFl0fS2CUQ0tRxaCbCMCRz6Djj66mfucehKQxv1XjBLwIDzgNKcFKS32z4jIdDvBd5kZYDWWEGQOR1U6OJ5O+WbXrdg5rkTgFqaICzb757QYxSSPgijAsZc+xE+VlWChKGEzKbiohwWSEB9DTlOMIpJflR0ysI60EVhgL/biycvX3RIYjKceRr5P5T+e9qfivJ4BJ+jC/XoUXXkZ5kirgZLY9TJmlYmtakxVQAUSxsSaYCKoPsMgQC0yULrOsm3Hme1gk7dO6Y9edm7Nq47LoqIQq6BDVkIPOUQNdoyZqIYIe0TN6Qa/Gk/FmvBsfs+iSUcwcoD8wPn8AsK2hJQ==</latexit>

�(�)<latexit sha1_base64="SY3GN5bhA+yFjkNa/mwA5q2eC80=">AAACGXicbVDLSsNAFJ34rPUVddlNsAh1YUlE0JUU3LisYB/QxDKZ3LRDJ5MwMxFq6MLv8APc6ie4E7eu/AJ/w2mahbYeGDiccy537vETRqWy7S9jaXlldW29tFHe3Nre2TX39tsyTgWBFolZLLo+lsAoh5aiikE3EYAjn0HHH11N/c49CEljfqvGCXgRHnAaUoKVlvpmxWU6HOC7zI2wGsoIMwYiq50cTyZ9s2rX7RzWInEKUkUFmn3z2w1ikkbAFWFYyp5jJ8rLsFCUMJiU3VRCgskID6CnKccRSC/Lj5hYR1oJrDAW+nFl5erviQxHUo4jXyfzn857U/FfTwCT9GF+vQovvIzyJFXAyWx7mDJLxda0JiugAohiY00wEVQfYJEhFpgoXWZZN+PM97BI2qd1x647N2fVxmXRUQlV0CGqIQedowa6Rk3UQgQ9omf0gl6NJ+PNeDc+ZtElo5g5QH9gfP4As+WhJw==</latexit><latexit sha1_base64="SY3GN5bhA+yFjkNa/mwA5q2eC80=">AAACGXicbVDLSsNAFJ34rPUVddlNsAh1YUlE0JUU3LisYB/QxDKZ3LRDJ5MwMxFq6MLv8APc6ie4E7eu/AJ/w2mahbYeGDiccy537vETRqWy7S9jaXlldW29tFHe3Nre2TX39tsyTgWBFolZLLo+lsAoh5aiikE3EYAjn0HHH11N/c49CEljfqvGCXgRHnAaUoKVlvpmxWU6HOC7zI2wGsoIMwYiq50cTyZ9s2rX7RzWInEKUkUFmn3z2w1ikkbAFWFYyp5jJ8rLsFCUMJiU3VRCgskID6CnKccRSC/Lj5hYR1oJrDAW+nFl5erviQxHUo4jXyfzn857U/FfTwCT9GF+vQovvIzyJFXAyWx7mDJLxda0JiugAohiY00wEVQfYJEhFpgoXWZZN+PM97BI2qd1x647N2fVxmXRUQlV0CGqIQedowa6Rk3UQgQ9omf0gl6NJ+PNeDc+ZtElo5g5QH9gfP4As+WhJw==</latexit><latexit sha1_base64="SY3GN5bhA+yFjkNa/mwA5q2eC80=">AAACGXicbVDLSsNAFJ34rPUVddlNsAh1YUlE0JUU3LisYB/QxDKZ3LRDJ5MwMxFq6MLv8APc6ie4E7eu/AJ/w2mahbYeGDiccy537vETRqWy7S9jaXlldW29tFHe3Nre2TX39tsyTgWBFolZLLo+lsAoh5aiikE3EYAjn0HHH11N/c49CEljfqvGCXgRHnAaUoKVlvpmxWU6HOC7zI2wGsoIMwYiq50cTyZ9s2rX7RzWInEKUkUFmn3z2w1ikkbAFWFYyp5jJ8rLsFCUMJiU3VRCgskID6CnKccRSC/Lj5hYR1oJrDAW+nFl5erviQxHUo4jXyfzn857U/FfTwCT9GF+vQovvIzyJFXAyWx7mDJLxda0JiugAohiY00wEVQfYJEhFpgoXWZZN+PM97BI2qd1x647N2fVxmXRUQlV0CGqIQedowa6Rk3UQgQ9omf0gl6NJ+PNeDc+ZtElo5g5QH9gfP4As+WhJw==</latexit><latexit sha1_base64="SY3GN5bhA+yFjkNa/mwA5q2eC80=">AAACGXicbVDLSsNAFJ34rPUVddlNsAh1YUlE0JUU3LisYB/QxDKZ3LRDJ5MwMxFq6MLv8APc6ie4E7eu/AJ/w2mahbYeGDiccy537vETRqWy7S9jaXlldW29tFHe3Nre2TX39tsyTgWBFolZLLo+lsAoh5aiikE3EYAjn0HHH11N/c49CEljfqvGCXgRHnAaUoKVlvpmxWU6HOC7zI2wGsoIMwYiq50cTyZ9s2rX7RzWInEKUkUFmn3z2w1ikkbAFWFYyp5jJ8rLsFCUMJiU3VRCgskID6CnKccRSC/Lj5hYR1oJrDAW+nFl5erviQxHUo4jXyfzn857U/FfTwCT9GF+vQovvIzyJFXAyWx7mDJLxda0JiugAohiY00wEVQfYJEhFpgoXWZZN+PM97BI2qd1x647N2fVxmXRUQlV0CGqIQedowa6Rk3UQgQ9omf0gl6NJ+PNeDc+ZtElo5g5QH9gfP4As+WhJw==</latexit>

g(+)

<latexit sha1_base64="77Lhl0RQm4avfvpdvLFgG7l0224=">AAACGnicbVDLSsNAFJ3UV62vqLhyEyxCRSiJCLqSghuXFewD2lgm09t26MwkzEyEGvInbtzqX7gTt278Cb/BaZqF1h64cDjnXjj3BBGjSrvul1VYWl5ZXSuulzY2t7Z37N29pgpjSaBBQhbKdoAVMCqgoalm0I4kYB4waAXj66nfegCpaCju9CQCn+OhoANKsDZSzz4Y3iddjvVIccwYyKRyepKmPbvsVt0Mzn/i5aSMctR79ne3H5KYg9CEYaU6nhtpP8FSU8IgLXVjBREmYzyEjqECc1B+ksVPnWOj9J1BKM0I7WTq74sEc6UmPDCbWdJ5byou9CQwRR9hkdeJ9eDST6iIYg2CzEIMYubo0Jn25PSpBKLZxBBMJDV/OGSEJSbatFkyBXnzdfwnzbOq51a92/Ny7SqvqogO0RGqIA9doBq6QXXUQAQl6Bm9oFfryXqz3q2P2WrBym/20R9Ynz/jw6Ef</latexit><latexit sha1_base64="77Lhl0RQm4avfvpdvLFgG7l0224=">AAACGnicbVDLSsNAFJ3UV62vqLhyEyxCRSiJCLqSghuXFewD2lgm09t26MwkzEyEGvInbtzqX7gTt278Cb/BaZqF1h64cDjnXjj3BBGjSrvul1VYWl5ZXSuulzY2t7Z37N29pgpjSaBBQhbKdoAVMCqgoalm0I4kYB4waAXj66nfegCpaCju9CQCn+OhoANKsDZSzz4Y3iddjvVIccwYyKRyepKmPbvsVt0Mzn/i5aSMctR79ne3H5KYg9CEYaU6nhtpP8FSU8IgLXVjBREmYzyEjqECc1B+ksVPnWOj9J1BKM0I7WTq74sEc6UmPDCbWdJ5byou9CQwRR9hkdeJ9eDST6iIYg2CzEIMYubo0Jn25PSpBKLZxBBMJDV/OGSEJSbatFkyBXnzdfwnzbOq51a92/Ny7SqvqogO0RGqIA9doBq6QXXUQAQl6Bm9oFfryXqz3q2P2WrBym/20R9Ynz/jw6Ef</latexit><latexit sha1_base64="77Lhl0RQm4avfvpdvLFgG7l0224=">AAACGnicbVDLSsNAFJ3UV62vqLhyEyxCRSiJCLqSghuXFewD2lgm09t26MwkzEyEGvInbtzqX7gTt278Cb/BaZqF1h64cDjnXjj3BBGjSrvul1VYWl5ZXSuulzY2t7Z37N29pgpjSaBBQhbKdoAVMCqgoalm0I4kYB4waAXj66nfegCpaCju9CQCn+OhoANKsDZSzz4Y3iddjvVIccwYyKRyepKmPbvsVt0Mzn/i5aSMctR79ne3H5KYg9CEYaU6nhtpP8FSU8IgLXVjBREmYzyEjqECc1B+ksVPnWOj9J1BKM0I7WTq74sEc6UmPDCbWdJ5byou9CQwRR9hkdeJ9eDST6iIYg2CzEIMYubo0Jn25PSpBKLZxBBMJDV/OGSEJSbatFkyBXnzdfwnzbOq51a92/Ny7SqvqogO0RGqIA9doBq6QXXUQAQl6Bm9oFfryXqz3q2P2WrBym/20R9Ynz/jw6Ef</latexit><latexit sha1_base64="77Lhl0RQm4avfvpdvLFgG7l0224=">AAACGnicbVDLSsNAFJ3UV62vqLhyEyxCRSiJCLqSghuXFewD2lgm09t26MwkzEyEGvInbtzqX7gTt278Cb/BaZqF1h64cDjnXjj3BBGjSrvul1VYWl5ZXSuulzY2t7Z37N29pgpjSaBBQhbKdoAVMCqgoalm0I4kYB4waAXj66nfegCpaCju9CQCn+OhoANKsDZSzz4Y3iddjvVIccwYyKRyepKmPbvsVt0Mzn/i5aSMctR79ne3H5KYg9CEYaU6nhtpP8FSU8IgLXVjBREmYzyEjqECc1B+ksVPnWOj9J1BKM0I7WTq74sEc6UmPDCbWdJ5byou9CQwRR9hkdeJ9eDST6iIYg2CzEIMYubo0Jn25PSpBKLZxBBMJDV/OGSEJSbatFkyBXnzdfwnzbOq51a92/Ny7SqvqogO0RGqIA9doBq6QXXUQAQl6Bm9oFfryXqz3q2P2WrBym/20R9Ynz/jw6Ef</latexit>

y(±)






y(+)

<latexit sha1_base64="Ag1remozIJoESEiHGl3iJPUVltQ=">AAACInicbVDLSsNAFJ3UV62vqEsRgkWoCCURQVdScOOygn1AE8tkctMOnTyYmQgxZOWvuHGrf+FOXAl+g9/gtM1C2x64cDjnXjj3uDGjQprml1ZaWl5ZXSuvVzY2t7Z39N29togSTqBFIhbxrosFMBpCS1LJoBtzwIHLoOOOrsd+5wG4oFF4J9MYnAAPQupTgqWS+vqhLSnzIEvz+8wOsByKADMGPKudnuR5X6+adXMCY55YBamiAs2+/mN7EUkCCCVhWIieZcbSyTCXlDDIK3YiIMZkhAfQUzTEAQgnm7yRG8dK8Qw/4mpCaUzUvxcZDoRIA1dtTpLOemNxoceBCfoIi7xeIv1LJ6NhnEgIyTSEnzBDRsa4L8OjHIhkqSKYcKr+MMgQc0ykarWiCrJm65gn7bO6Zdat2/Nq46qoqowO0BGqIQtdoAa6QU3UQgQ9oRf0it60Z+1d+9A+p6slrbjZR/+gff8CBdek5w==</latexit><latexit sha1_base64="Ag1remozIJoESEiHGl3iJPUVltQ=">AAACInicbVDLSsNAFJ3UV62vqEsRgkWoCCURQVdScOOygn1AE8tkctMOnTyYmQgxZOWvuHGrf+FOXAl+g9/gtM1C2x64cDjnXjj3uDGjQprml1ZaWl5ZXSuvVzY2t7Z39N29togSTqBFIhbxrosFMBpCS1LJoBtzwIHLoOOOrsd+5wG4oFF4J9MYnAAPQupTgqWS+vqhLSnzIEvz+8wOsByKADMGPKudnuR5X6+adXMCY55YBamiAs2+/mN7EUkCCCVhWIieZcbSyTCXlDDIK3YiIMZkhAfQUzTEAQgnm7yRG8dK8Qw/4mpCaUzUvxcZDoRIA1dtTpLOemNxoceBCfoIi7xeIv1LJ6NhnEgIyTSEnzBDRsa4L8OjHIhkqSKYcKr+MMgQc0ykarWiCrJm65gn7bO6Zdat2/Nq46qoqowO0BGqIQtdoAa6QU3UQgQ9oRf0it60Z+1d+9A+p6slrbjZR/+gff8CBdek5w==</latexit><latexit sha1_base64="Ag1remozIJoESEiHGl3iJPUVltQ=">AAACInicbVDLSsNAFJ3UV62vqEsRgkWoCCURQVdScOOygn1AE8tkctMOnTyYmQgxZOWvuHGrf+FOXAl+g9/gtM1C2x64cDjnXjj3uDGjQprml1ZaWl5ZXSuvVzY2t7Z39N29togSTqBFIhbxrosFMBpCS1LJoBtzwIHLoOOOrsd+5wG4oFF4J9MYnAAPQupTgqWS+vqhLSnzIEvz+8wOsByKADMGPKudnuR5X6+adXMCY55YBamiAs2+/mN7EUkCCCVhWIieZcbSyTCXlDDIK3YiIMZkhAfQUzTEAQgnm7yRG8dK8Qw/4mpCaUzUvxcZDoRIA1dtTpLOemNxoceBCfoIi7xeIv1LJ6NhnEgIyTSEnzBDRsa4L8OjHIhkqSKYcKr+MMgQc0ykarWiCrJm65gn7bO6Zdat2/Nq46qoqowO0BGqIQtdoAa6QU3UQgQ9oRf0it60Z+1d+9A+p6slrbjZR/+gff8CBdek5w==</latexit><latexit sha1_base64="Ag1remozIJoESEiHGl3iJPUVltQ=">AAACInicbVDLSsNAFJ3UV62vqEsRgkWoCCURQVdScOOygn1AE8tkctMOnTyYmQgxZOWvuHGrf+FOXAl+g9/gtM1C2x64cDjnXjj3uDGjQprml1ZaWl5ZXSuvVzY2t7Z39N29togSTqBFIhbxrosFMBpCS1LJoBtzwIHLoOOOrsd+5wG4oFF4J9MYnAAPQupTgqWS+vqhLSnzIEvz+8wOsByKADMGPKudnuR5X6+adXMCY55YBamiAs2+/mN7EUkCCCVhWIieZcbSyTCXlDDIK3YiIMZkhAfQUzTEAQgnm7yRG8dK8Qw/4mpCaUzUvxcZDoRIA1dtTpLOemNxoceBCfoIi7xeIv1LJ6NhnEgIyTSEnzBDRsa4L8OjHIhkqSKYcKr+MMgQc0ykarWiCrJm65gn7bO6Zdat2/Nq46qoqowO0BGqIQtdoAa6QU3UQgQ9oRf0it60Z+1d+9A+p6slrbjZR/+gff8CBdek5w==</latexit>



y(±)


g(�)

<latexit sha1_base64="jVLRleK787/Z2+P+899rA32Gnew=">AAACGnicbVDLSsNAFJ3UV62vqLhyEyxCXVgSEXQlBTcuK9gHtLFMprft0JlJmJkINeRP3LjVv3Anbt34E36D0zQLrT1w4XDOvXDuCSJGlXbdL6uwtLyyulZcL21sbm3v2Lt7TRXGkkCDhCyU7QArYFRAQ1PNoB1JwDxg0ArG11O/9QBS0VDc6UkEPsdDQQeUYG2knn0wvE+6HOuR4pgxkEnl9CRNe3bZrboZnP/Ey0kZ5aj37O9uPyQxB6EJw0p1PDfSfoKlpoRBWurGCiJMxngIHUMF5qD8JIufOsdG6TuDUJoR2snU3xcJ5kpNeGA2s6Tz3lRc6Elgij7CIq8T68Gln1ARxRoEmYUYxMzRoTPtyelTCUSziSGYSGr+cMgIS0y0abNkCvLm6/hPmmdVz616t+fl2lVeVREdoiNUQR66QDV0g+qogQhK0DN6Qa/Wk/VmvVsfs9WCld/soz+wPn8A5wmhIQ==</latexit><latexit sha1_base64="jVLRleK787/Z2+P+899rA32Gnew=">AAACGnicbVDLSsNAFJ3UV62vqLhyEyxCXVgSEXQlBTcuK9gHtLFMprft0JlJmJkINeRP3LjVv3Anbt34E36D0zQLrT1w4XDOvXDuCSJGlXbdL6uwtLyyulZcL21sbm3v2Lt7TRXGkkCDhCyU7QArYFRAQ1PNoB1JwDxg0ArG11O/9QBS0VDc6UkEPsdDQQeUYG2knn0wvE+6HOuR4pgxkEnl9CRNe3bZrboZnP/Ey0kZ5aj37O9uPyQxB6EJw0p1PDfSfoKlpoRBWurGCiJMxngIHUMF5qD8JIufOsdG6TuDUJoR2snU3xcJ5kpNeGA2s6Tz3lRc6Elgij7CIq8T68Gln1ARxRoEmYUYxMzRoTPtyelTCUSziSGYSGr+cMgIS0y0abNkCvLm6/hPmmdVz616t+fl2lVeVREdoiNUQR66QDV0g+qogQhK0DN6Qa/Wk/VmvVsfs9WCld/soz+wPn8A5wmhIQ==</latexit><latexit sha1_base64="jVLRleK787/Z2+P+899rA32Gnew=">AAACGnicbVDLSsNAFJ3UV62vqLhyEyxCXVgSEXQlBTcuK9gHtLFMprft0JlJmJkINeRP3LjVv3Anbt34E36D0zQLrT1w4XDOvXDuCSJGlXbdL6uwtLyyulZcL21sbm3v2Lt7TRXGkkCDhCyU7QArYFRAQ1PNoB1JwDxg0ArG11O/9QBS0VDc6UkEPsdDQQeUYG2knn0wvE+6HOuR4pgxkEnl9CRNe3bZrboZnP/Ey0kZ5aj37O9uPyQxB6EJw0p1PDfSfoKlpoRBWurGCiJMxngIHUMF5qD8JIufOsdG6TuDUJoR2snU3xcJ5kpNeGA2s6Tz3lRc6Elgij7CIq8T68Gln1ARxRoEmYUYxMzRoTPtyelTCUSziSGYSGr+cMgIS0y0abNkCvLm6/hPmmdVz616t+fl2lVeVREdoiNUQR66QDV0g+qogQhK0DN6Qa/Wk/VmvVsfs9WCld/soz+wPn8A5wmhIQ==</latexit><latexit sha1_base64="jVLRleK787/Z2+P+899rA32Gnew=">AAACGnicbVDLSsNAFJ3UV62vqLhyEyxCXVgSEXQlBTcuK9gHtLFMprft0JlJmJkINeRP3LjVv3Anbt34E36D0zQLrT1w4XDOvXDuCSJGlXbdL6uwtLyyulZcL21sbm3v2Lt7TRXGkkCDhCyU7QArYFRAQ1PNoB1JwDxg0ArG11O/9QBS0VDc6UkEPsdDQQeUYG2knn0wvE+6HOuR4pgxkEnl9CRNe3bZrboZnP/Ey0kZ5aj37O9uPyQxB6EJw0p1PDfSfoKlpoRBWurGCiJMxngIHUMF5qD8JIufOsdG6TuDUJoR2snU3xcJ5kpNeGA2s6Tz3lRc6Elgij7CIq8T68Gln1ARxRoEmYUYxMzRoTPtyelTCUSziSGYSGr+cMgIS0y0abNkCvLm6/hPmmdVz616t+fl2lVeVREdoiNUQR66QDV0g+qogQhK0DN6Qa/Wk/VmvVsfs9WCld/soz+wPn8A5wmhIQ==</latexit>



observed<latexit sha1_base64="UheCCfyZS1wyN8nNAp+mhrKcVGo=">AAACC3icbZDNSgMxFIXv+FvrX9Wlm2ARXJWZbtRdwY3LCvYH2qFkMnfa0MxkSDKFWvoIbtzqW7gTtz6EL+EzmLaz0LYHAodz7oWbL0gF18Z1v52Nza3tnd3CXnH/4PDouHRy2tQyUwwbTAqp2gHVKHiCDcONwHaqkMaBwFYwvJv1rREqzWXyaMYp+jHtJzzijBobtWWgUY0w7JXKbsWdi6waLzdlyFXvlX66oWRZjIlhgmrd8dzU+BOqDGcCp8VupjGlbEj72LE2oTFqfzK/d0oubRKSSCr7EkPm6d+NCY21HseBnYypGejlbhau7RQKzZ9wXdfJTHTjT3iSZgYTtjgiygQxkszAkJArZEaMraFMcfsPwgZUUWYsvqIF5C3jWDXNasVzK95DtVy7zVEV4Bwu4Ao8uIYa3EMdGsBAwAu8wpvz7Lw7H87nYnTDyXfO4J+cr19s4JuY</latexit><latexit sha1_base64="UheCCfyZS1wyN8nNAp+mhrKcVGo=">AAACC3icbZDNSgMxFIXv+FvrX9Wlm2ARXJWZbtRdwY3LCvYH2qFkMnfa0MxkSDKFWvoIbtzqW7gTtz6EL+EzmLaz0LYHAodz7oWbL0gF18Z1v52Nza3tnd3CXnH/4PDouHRy2tQyUwwbTAqp2gHVKHiCDcONwHaqkMaBwFYwvJv1rREqzWXyaMYp+jHtJzzijBobtWWgUY0w7JXKbsWdi6waLzdlyFXvlX66oWRZjIlhgmrd8dzU+BOqDGcCp8VupjGlbEj72LE2oTFqfzK/d0oubRKSSCr7EkPm6d+NCY21HseBnYypGejlbhau7RQKzZ9wXdfJTHTjT3iSZgYTtjgiygQxkszAkJArZEaMraFMcfsPwgZUUWYsvqIF5C3jWDXNasVzK95DtVy7zVEV4Bwu4Ao8uIYa3EMdGsBAwAu8wpvz7Lw7H87nYnTDyXfO4J+cr19s4JuY</latexit><latexit sha1_base64="UheCCfyZS1wyN8nNAp+mhrKcVGo=">AAACC3icbZDNSgMxFIXv+FvrX9Wlm2ARXJWZbtRdwY3LCvYH2qFkMnfa0MxkSDKFWvoIbtzqW7gTtz6EL+EzmLaz0LYHAodz7oWbL0gF18Z1v52Nza3tnd3CXnH/4PDouHRy2tQyUwwbTAqp2gHVKHiCDcONwHaqkMaBwFYwvJv1rREqzWXyaMYp+jHtJzzijBobtWWgUY0w7JXKbsWdi6waLzdlyFXvlX66oWRZjIlhgmrd8dzU+BOqDGcCp8VupjGlbEj72LE2oTFqfzK/d0oubRKSSCr7EkPm6d+NCY21HseBnYypGejlbhau7RQKzZ9wXdfJTHTjT3iSZgYTtjgiygQxkszAkJArZEaMraFMcfsPwgZUUWYsvqIF5C3jWDXNasVzK95DtVy7zVEV4Bwu4Ao8uIYa3EMdGsBAwAu8wpvz7Lw7H87nYnTDyXfO4J+cr19s4JuY</latexit><latexit sha1_base64="UheCCfyZS1wyN8nNAp+mhrKcVGo=">AAACC3icbZDNSgMxFIXv+FvrX9Wlm2ARXJWZbtRdwY3LCvYH2qFkMnfa0MxkSDKFWvoIbtzqW7gTtz6EL+EzmLaz0LYHAodz7oWbL0gF18Z1v52Nza3tnd3CXnH/4PDouHRy2tQyUwwbTAqp2gHVKHiCDcONwHaqkMaBwFYwvJv1rREqzWXyaMYp+jHtJzzijBobtWWgUY0w7JXKbsWdi6waLzdlyFXvlX66oWRZjIlhgmrd8dzU+BOqDGcCp8VupjGlbEj72LE2oTFqfzK/d0oubRKSSCr7EkPm6d+NCY21HseBnYypGejlbhau7RQKzZ9wXdfJTHTjT3iSZgYTtjgiygQxkszAkJArZEaMraFMcfsPwgZUUWYsvqIF5C3jWDXNasVzK95DtVy7zVEV4Bwu4Ao8uIYa3EMdGsBAwAu8wpvz7Lw7H87nYnTDyXfO4J+cr19s4JuY</latexit>

deterministic<latexit sha1_base64="QcCpJIszCaaQ0MwUZ2XhzLsUrRU=">AAACEHicbVC7TsNAEFyHVwivACWNRYREFdlpgC4SDWWQyENKrOh83iSn3J3N3RkpWPkJGlr4CzpEyx/wE3wDl8QFhIy00mhm9253woQzbTzvyymsrW9sbhW3Szu7e/sH5cOjlo5TRbFJYx6rTkg0ciaxaZjh2EkUEhFybIfj65nffkClWSzvzCTBQJChZANGibFSEKFBJZi0HzHaL1e8qjeH+5/4OalAjka//N2LYpoKlIZyonXX9xITZETZxzhOS71UY0LomAyxa6kkAnWQzZeeumdWidxBrGxJ487V3xMZEVpPRGg7BTEjvezNxJWeQq7ZI67yuqkZXAYZk0lqUNLFEoOUuyZ2Z+m4EVNIDZ9YQqhi9g6Xjogi1MakSzYgfzmO/6RVq/pe1b+tVepXeVRFOIFTOAcfLqAON9CAJlC4h2d4gVfnyXlz3p2PRWvByWeO4Q+czx+leJ3k</latexit><latexit sha1_base64="QcCpJIszCaaQ0MwUZ2XhzLsUrRU=">AAACEHicbVC7TsNAEFyHVwivACWNRYREFdlpgC4SDWWQyENKrOh83iSn3J3N3RkpWPkJGlr4CzpEyx/wE3wDl8QFhIy00mhm9253woQzbTzvyymsrW9sbhW3Szu7e/sH5cOjlo5TRbFJYx6rTkg0ciaxaZjh2EkUEhFybIfj65nffkClWSzvzCTBQJChZANGibFSEKFBJZi0HzHaL1e8qjeH+5/4OalAjka//N2LYpoKlIZyonXX9xITZETZxzhOS71UY0LomAyxa6kkAnWQzZeeumdWidxBrGxJ487V3xMZEVpPRGg7BTEjvezNxJWeQq7ZI67yuqkZXAYZk0lqUNLFEoOUuyZ2Z+m4EVNIDZ9YQqhi9g6Xjogi1MakSzYgfzmO/6RVq/pe1b+tVepXeVRFOIFTOAcfLqAON9CAJlC4h2d4gVfnyXlz3p2PRWvByWeO4Q+czx+leJ3k</latexit><latexit sha1_base64="QcCpJIszCaaQ0MwUZ2XhzLsUrRU=">AAACEHicbVC7TsNAEFyHVwivACWNRYREFdlpgC4SDWWQyENKrOh83iSn3J3N3RkpWPkJGlr4CzpEyx/wE3wDl8QFhIy00mhm9253woQzbTzvyymsrW9sbhW3Szu7e/sH5cOjlo5TRbFJYx6rTkg0ciaxaZjh2EkUEhFybIfj65nffkClWSzvzCTBQJChZANGibFSEKFBJZi0HzHaL1e8qjeH+5/4OalAjka//N2LYpoKlIZyonXX9xITZETZxzhOS71UY0LomAyxa6kkAnWQzZeeumdWidxBrGxJ487V3xMZEVpPRGg7BTEjvezNxJWeQq7ZI67yuqkZXAYZk0lqUNLFEoOUuyZ2Z+m4EVNIDZ9YQqhi9g6Xjogi1MakSzYgfzmO/6RVq/pe1b+tVepXeVRFOIFTOAcfLqAON9CAJlC4h2d4gVfnyXlz3p2PRWvByWeO4Q+czx+leJ3k</latexit><latexit sha1_base64="QcCpJIszCaaQ0MwUZ2XhzLsUrRU=">AAACEHicbVC7TsNAEFyHVwivACWNRYREFdlpgC4SDWWQyENKrOh83iSn3J3N3RkpWPkJGlr4CzpEyx/wE3wDl8QFhIy00mhm9253woQzbTzvyymsrW9sbhW3Szu7e/sH5cOjlo5TRbFJYx6rTkg0ciaxaZjh2EkUEhFybIfj65nffkClWSzvzCTBQJChZANGibFSEKFBJZi0HzHaL1e8qjeH+5/4OalAjka//N2LYpoKlIZyonXX9xITZETZxzhOS71UY0LomAyxa6kkAnWQzZeeumdWidxBrGxJ487V3xMZEVpPRGg7BTEjvezNxJWeQq7ZI67yuqkZXAYZk0lqUNLFEoOUuyZ2Z+m4EVNIDZ9YQqhi9g6Xjogi1MakSzYgfzmO/6RVq/pe1b+tVepXeVRFOIFTOAcfLqAON9CAJlC4h2d4gVfnyXlz3p2PRWvByWeO4Q+czx+leJ3k</latexit>

random<latexit sha1_base64="JMYIWor/B4JN83rBOajvBPPlkdM=">AAACCXicbZC7SgNBFIZn4y3GW9TSZjAIVmE3jdoFbCwjmAskIczOnk3GzGWZmRXikiewsdW3sBNbn8KX8BmcJFtozA8DP/9/Dpz5woQzY33/yyusrW9sbhW3Szu7e/sH5cOjllGpptCkiivdCYkBziQ0LbMcOokGIkIO7XB8PevbD6ANU/LOThLoCzKULGaUWBe1NJGREoNyxa/6c+H/JshNBeVqDMrfvUjRVIC0lBNjuoGf2H5GtGWUw7TUSw0khI7JELrOSiLA9LP5tVN85pIIx0q7Jy2ep783MiKMmYjQTQpiR2a5m4UrOw3csEdY1XVTG1/2MyaT1IKkiyPilGOr8AwLjpgGavnEGUI1c//AdEQ0odbBKzlAwTKO/6ZVqwZ+NbitVepXOaoiOkGn6BwF6ALV0Q1qoCai6B49oxf06j15b96797EYLXj5zjH6I+/zB7x8mqs=</latexit><latexit sha1_base64="JMYIWor/B4JN83rBOajvBPPlkdM=">AAACCXicbZC7SgNBFIZn4y3GW9TSZjAIVmE3jdoFbCwjmAskIczOnk3GzGWZmRXikiewsdW3sBNbn8KX8BmcJFtozA8DP/9/Dpz5woQzY33/yyusrW9sbhW3Szu7e/sH5cOjllGpptCkiivdCYkBziQ0LbMcOokGIkIO7XB8PevbD6ANU/LOThLoCzKULGaUWBe1NJGREoNyxa/6c+H/JshNBeVqDMrfvUjRVIC0lBNjuoGf2H5GtGWUw7TUSw0khI7JELrOSiLA9LP5tVN85pIIx0q7Jy2ep783MiKMmYjQTQpiR2a5m4UrOw3csEdY1XVTG1/2MyaT1IKkiyPilGOr8AwLjpgGavnEGUI1c//AdEQ0odbBKzlAwTKO/6ZVqwZ+NbitVepXOaoiOkGn6BwF6ALV0Q1qoCai6B49oxf06j15b96797EYLXj5zjH6I+/zB7x8mqs=</latexit><latexit sha1_base64="JMYIWor/B4JN83rBOajvBPPlkdM=">AAACCXicbZC7SgNBFIZn4y3GW9TSZjAIVmE3jdoFbCwjmAskIczOnk3GzGWZmRXikiewsdW3sBNbn8KX8BmcJFtozA8DP/9/Dpz5woQzY33/yyusrW9sbhW3Szu7e/sH5cOjllGpptCkiivdCYkBziQ0LbMcOokGIkIO7XB8PevbD6ANU/LOThLoCzKULGaUWBe1NJGREoNyxa/6c+H/JshNBeVqDMrfvUjRVIC0lBNjuoGf2H5GtGWUw7TUSw0khI7JELrOSiLA9LP5tVN85pIIx0q7Jy2ep783MiKMmYjQTQpiR2a5m4UrOw3csEdY1XVTG1/2MyaT1IKkiyPilGOr8AwLjpgGavnEGUI1c//AdEQ0odbBKzlAwTKO/6ZVqwZ+NbitVepXOaoiOkGn6BwF6ALV0Q1qoCai6B49oxf06j15b96797EYLXj5zjH6I+/zB7x8mqs=</latexit><latexit sha1_base64="JMYIWor/B4JN83rBOajvBPPlkdM=">AAACCXicbZC7SgNBFIZn4y3GW9TSZjAIVmE3jdoFbCwjmAskIczOnk3GzGWZmRXikiewsdW3sBNbn8KX8BmcJFtozA8DP/9/Dpz5woQzY33/yyusrW9sbhW3Szu7e/sH5cOjllGpptCkiivdCYkBziQ0LbMcOokGIkIO7XB8PevbD6ANU/LOThLoCzKULGaUWBe1NJGREoNyxa/6c+H/JshNBeVqDMrfvUjRVIC0lBNjuoGf2H5GtGWUw7TUSw0khI7JELrOSiLA9LP5tVN85pIIx0q7Jy2ep783MiKMmYjQTQpiR2a5m4UrOw3csEdY1XVTG1/2MyaT1IKkiyPilGOr8AwLjpgGavnEGUI1c//AdEQ0odbBKzlAwTKO/6ZVqwZ+NbitVepXOaoiOkGn6BwF6ALV0Q1qoCai6B49oxf06j15b96797EYLXj5zjH6I+/zB7x8mqs=</latexit>fixed

<latexit sha1_base64="snwC3knKgHk2ZluvKNBW7wkTd5M=">AAACCHicbZDNSsNAFIVv/K31r+rSTbAIrkrSjboruHFZwbSFNpTJ5KYdOpmEmYlYQ1/AjVt9C3fi1rfwJXwGp20W2vbAwOGce+HOF6ScKe0439ba+sbm1nZpp7y7t39wWDk6bqkkkxQ9mvBEdgKikDOBnmaaYyeVSOKAYzsY3Uz79gNKxRJxr8cp+jEZCBYxSrSJvIg9YtivVJ2aM5O9bNzCVKFQs1/56YUJzWIUmnKiVNd1Uu3nRGpGOU7KvUxhSuiIDLBrrCAxKj+fHTuxz00S2lEizRPanqV/N3ISKzWOAzMZEz1Ui900XNlJ5Io94aqum+noys+ZSDONgs6PiDJu68SeUrFDJpFqPjaGUMnMP2w6JJJQbdiVDSB3EceyadVrrlNz7+rVxnWBqgSncAYX4MIlNOAWmuABBQYv8Apv1rP1bn1Yn/PRNavYOYF/sr5+Ad2umjA=</latexit><latexit sha1_base64="snwC3knKgHk2ZluvKNBW7wkTd5M=">AAACCHicbZDNSsNAFIVv/K31r+rSTbAIrkrSjboruHFZwbSFNpTJ5KYdOpmEmYlYQ1/AjVt9C3fi1rfwJXwGp20W2vbAwOGce+HOF6ScKe0439ba+sbm1nZpp7y7t39wWDk6bqkkkxQ9mvBEdgKikDOBnmaaYyeVSOKAYzsY3Uz79gNKxRJxr8cp+jEZCBYxSrSJvIg9YtivVJ2aM5O9bNzCVKFQs1/56YUJzWIUmnKiVNd1Uu3nRGpGOU7KvUxhSuiIDLBrrCAxKj+fHTuxz00S2lEizRPanqV/N3ISKzWOAzMZEz1Ui900XNlJ5Io94aqum+noys+ZSDONgs6PiDJu68SeUrFDJpFqPjaGUMnMP2w6JJJQbdiVDSB3EceyadVrrlNz7+rVxnWBqgSncAYX4MIlNOAWmuABBQYv8Apv1rP1bn1Yn/PRNavYOYF/sr5+Ad2umjA=</latexit><latexit sha1_base64="snwC3knKgHk2ZluvKNBW7wkTd5M=">AAACCHicbZDNSsNAFIVv/K31r+rSTbAIrkrSjboruHFZwbSFNpTJ5KYdOpmEmYlYQ1/AjVt9C3fi1rfwJXwGp20W2vbAwOGce+HOF6ScKe0439ba+sbm1nZpp7y7t39wWDk6bqkkkxQ9mvBEdgKikDOBnmaaYyeVSOKAYzsY3Uz79gNKxRJxr8cp+jEZCBYxSrSJvIg9YtivVJ2aM5O9bNzCVKFQs1/56YUJzWIUmnKiVNd1Uu3nRGpGOU7KvUxhSuiIDLBrrCAxKj+fHTuxz00S2lEizRPanqV/N3ISKzWOAzMZEz1Ui900XNlJ5Io94aqum+noys+ZSDONgs6PiDJu68SeUrFDJpFqPjaGUMnMP2w6JJJQbdiVDSB3EceyadVrrlNz7+rVxnWBqgSncAYX4MIlNOAWmuABBQYv8Apv1rP1bn1Yn/PRNavYOYF/sr5+Ad2umjA=</latexit><latexit sha1_base64="snwC3knKgHk2ZluvKNBW7wkTd5M=">AAACCHicbZDNSsNAFIVv/K31r+rSTbAIrkrSjboruHFZwbSFNpTJ5KYdOpmEmYlYQ1/AjVt9C3fi1rfwJXwGp20W2vbAwOGce+HOF6ScKe0439ba+sbm1nZpp7y7t39wWDk6bqkkkxQ9mvBEdgKikDOBnmaaYyeVSOKAYzsY3Uz79gNKxRJxr8cp+jEZCBYxSrSJvIg9YtivVJ2aM5O9bNzCVKFQs1/56YUJzWIUmnKiVNd1Uu3nRGpGOU7KvUxhSuiIDLBrrCAxKj+fHTuxz00S2lEizRPanqV/N3ISKzWOAzMZEz1Ui900XNlJ5Io94aqum+noys+ZSDONgs6PiDJu68SeUrFDJpFqPjaGUMnMP2w6JJJQbdiVDSB3EceyadVrrlNz7+rVxnWBqgSncAYX4MIlNOAWmuABBQYv8Apv1rP1bn1Yn/PRNavYOYF/sr5+Ad2umjA=</latexit>

Process 2<latexit sha1_base64="vbW0jF95gvVCOtsd+ZsH81+1tqs=">AAACInicbVDLSsNAFJ34rPEVdSnCYBFclaQbuyy4cVnBPqAJZTK9aYdOJmFmItbQlb/ixq3+hTtxJfgNfoPTNgtte+DC4Zx7L/eeMOVMadf9stbWNza3tks79u7e/sGhc3TcUkkmKTRpwhPZCYkCzgQ0NdMcOqkEEocc2uHoeuq370Eqlog7PU4hiMlAsIhRoo3Uc858CkKDZGJg+xoedBjlDZlQUApXJz2n7FbcGfAy8QpSRgUaPefH7yc0i81OyolSXc9NdZATqRnlMLH9TEFK6IgMoGuoIDGoIJ+9McEXRunjKJGmhMYz9e9ETmKlxnFoOmOih2rRm4orPQlcsUdY5XUzHdWCnIk00yDo/Igo41gneJoX7jMJVPOxIYRKZv7AdEgkoSY0ZZuAvMU4lkmrWvHcindbLddrRVQldIrO0SXy0BWqoxvUQE1E0RN6Qa/ozXq23q0P63PeumYVMyfoH6zvX1EUpHc=</latexit><latexit sha1_base64="vbW0jF95gvVCOtsd+ZsH81+1tqs=">AAACInicbVDLSsNAFJ34rPEVdSnCYBFclaQbuyy4cVnBPqAJZTK9aYdOJmFmItbQlb/ixq3+hTtxJfgNfoPTNgtte+DC4Zx7L/eeMOVMadf9stbWNza3tks79u7e/sGhc3TcUkkmKTRpwhPZCYkCzgQ0NdMcOqkEEocc2uHoeuq370Eqlog7PU4hiMlAsIhRoo3Uc858CkKDZGJg+xoedBjlDZlQUApXJz2n7FbcGfAy8QpSRgUaPefH7yc0i81OyolSXc9NdZATqRnlMLH9TEFK6IgMoGuoIDGoIJ+9McEXRunjKJGmhMYz9e9ETmKlxnFoOmOih2rRm4orPQlcsUdY5XUzHdWCnIk00yDo/Igo41gneJoX7jMJVPOxIYRKZv7AdEgkoSY0ZZuAvMU4lkmrWvHcindbLddrRVQldIrO0SXy0BWqoxvUQE1E0RN6Qa/ozXq23q0P63PeumYVMyfoH6zvX1EUpHc=</latexit><latexit sha1_base64="vbW0jF95gvVCOtsd+ZsH81+1tqs=">AAACInicbVDLSsNAFJ34rPEVdSnCYBFclaQbuyy4cVnBPqAJZTK9aYdOJmFmItbQlb/ixq3+hTtxJfgNfoPTNgtte+DC4Zx7L/eeMOVMadf9stbWNza3tks79u7e/sGhc3TcUkkmKTRpwhPZCYkCzgQ0NdMcOqkEEocc2uHoeuq370Eqlog7PU4hiMlAsIhRoo3Uc858CkKDZGJg+xoedBjlDZlQUApXJz2n7FbcGfAy8QpSRgUaPefH7yc0i81OyolSXc9NdZATqRnlMLH9TEFK6IgMoGuoIDGoIJ+9McEXRunjKJGmhMYz9e9ETmKlxnFoOmOih2rRm4orPQlcsUdY5XUzHdWCnIk00yDo/Igo41gneJoX7jMJVPOxIYRKZv7AdEgkoSY0ZZuAvMU4lkmrWvHcindbLddrRVQldIrO0SXy0BWqoxvUQE1E0RN6Qa/ozXq23q0P63PeumYVMyfoH6zvX1EUpHc=</latexit><latexit sha1_base64="vbW0jF95gvVCOtsd+ZsH81+1tqs=">AAACInicbVDLSsNAFJ34rPEVdSnCYBFclaQbuyy4cVnBPqAJZTK9aYdOJmFmItbQlb/ixq3+hTtxJfgNfoPTNgtte+DC4Zx7L/eeMOVMadf9stbWNza3tks79u7e/sGhc3TcUkkmKTRpwhPZCYkCzgQ0NdMcOqkEEocc2uHoeuq370Eqlog7PU4hiMlAsIhRoo3Uc858CkKDZGJg+xoedBjlDZlQUApXJz2n7FbcGfAy8QpSRgUaPefH7yc0i81OyolSXc9NdZATqRnlMLH9TEFK6IgMoGuoIDGoIJ+9McEXRunjKJGmhMYz9e9ETmKlxnFoOmOih2rRm4orPQlcsUdY5XUzHdWCnIk00yDo/Igo41gneJoX7jMJVPOxIYRKZv7AdEgkoSY0ZZuAvMU4lkmrWvHcindbLddrRVQldIrO0SXy0BWqoxvUQE1E0RN6Qa/ozXq23q0P63PeumYVMyfoH6zvX1EUpHc=</latexit>Process 1

<latexit sha1_base64="MYcfOc2RAVFvCMSZM84sQYqsFPc=">AAACInicbVDLSsNAFJ34rPEVdSnCYBFclaQbdVdw47KCfUATymR60w6dTMLMRKyhK3/FjVv9C3fiSvAb/AanbRba9sCFwzn3Xu49YcqZ0q77Za2srq1vbJa27O2d3b195+CwqZJMUmjQhCeyHRIFnAloaKY5tFMJJA45tMLh9cRv3YNULBF3epRCEJO+YBGjRBup65z4FIQGyUTf9jU86DDK6zKhoBT2xl2n7FbcKfAi8QpSRgXqXefH7yU0i81OyolSHc9NdZATqRnlMLb9TEFK6JD0oWOoIDGoIJ++McZnRunhKJGmhMZT9e9ETmKlRnFoOmOiB2rem4hLPQlcsUdY5nUyHV0GORNppkHQ2RFRxrFO8CQv3GMSqOYjQwiVzPyB6YBIQk1oyjYBefNxLJJmteK5Fe+2Wq5dFVGV0DE6RefIQxeohm5QHTUQRU/oBb2iN+vZerc+rM9Z64pVzByhf7C+fwFPwKR3</latexit><latexit sha1_base64="MYcfOc2RAVFvCMSZM84sQYqsFPc=">AAACInicbVDLSsNAFJ34rPEVdSnCYBFclaQbdVdw47KCfUATymR60w6dTMLMRKyhK3/FjVv9C3fiSvAb/AanbRba9sCFwzn3Xu49YcqZ0q77Za2srq1vbJa27O2d3b195+CwqZJMUmjQhCeyHRIFnAloaKY5tFMJJA45tMLh9cRv3YNULBF3epRCEJO+YBGjRBup65z4FIQGyUTf9jU86DDK6zKhoBT2xl2n7FbcKfAi8QpSRgXqXefH7yU0i81OyolSHc9NdZATqRnlMLb9TEFK6JD0oWOoIDGoIJ++McZnRunhKJGmhMZT9e9ETmKlRnFoOmOiB2rem4hLPQlcsUdY5nUyHV0GORNppkHQ2RFRxrFO8CQv3GMSqOYjQwiVzPyB6YBIQk1oyjYBefNxLJJmteK5Fe+2Wq5dFVGV0DE6RefIQxeohm5QHTUQRU/oBb2iN+vZerc+rM9Z64pVzByhf7C+fwFPwKR3</latexit><latexit sha1_base64="MYcfOc2RAVFvCMSZM84sQYqsFPc=">AAACInicbVDLSsNAFJ34rPEVdSnCYBFclaQbdVdw47KCfUATymR60w6dTMLMRKyhK3/FjVv9C3fiSvAb/AanbRba9sCFwzn3Xu49YcqZ0q77Za2srq1vbJa27O2d3b195+CwqZJMUmjQhCeyHRIFnAloaKY5tFMJJA45tMLh9cRv3YNULBF3epRCEJO+YBGjRBup65z4FIQGyUTf9jU86DDK6zKhoBT2xl2n7FbcKfAi8QpSRgXqXefH7yU0i81OyolSHc9NdZATqRnlMLb9TEFK6JD0oWOoIDGoIJ++McZnRunhKJGmhMZT9e9ETmKlRnFoOmOiB2rem4hLPQlcsUdY5nUyHV0GORNppkHQ2RFRxrFO8CQv3GMSqOYjQwiVzPyB6YBIQk1oyjYBefNxLJJmteK5Fe+2Wq5dFVGV0DE6RefIQxeohm5QHTUQRU/oBb2iN+vZerc+rM9Z64pVzByhf7C+fwFPwKR3</latexit><latexit sha1_base64="MYcfOc2RAVFvCMSZM84sQYqsFPc=">AAACInicbVDLSsNAFJ34rPEVdSnCYBFclaQbdVdw47KCfUATymR60w6dTMLMRKyhK3/FjVv9C3fiSvAb/AanbRba9sCFwzn3Xu49YcqZ0q77Za2srq1vbJa27O2d3b195+CwqZJMUmjQhCeyHRIFnAloaKY5tFMJJA45tMLh9cRv3YNULBF3epRCEJO+YBGjRBup65z4FIQGyUTf9jU86DDK6zKhoBT2xl2n7FbcKfAi8QpSRgXqXefH7yU0i81OyolSHc9NdZATqRnlMLb9TEFK6JD0oWOoIDGoIJ++McZnRunhKJGmhMZT9e9ETmKlRnFoOmOiB2rem4hLPQlcsUdY5nUyHV0GORNppkHQ2RFRxrFO8CQv3GMSqOYjQwiVzPyB6YBIQk1oyjYBefNxLJJmteK5Fe+2Wq5dFVGV0DE6RefIQxeohm5QHTUQRU/oBb2iN+vZerc+rM9Z64pVzByhf7C+fwFPwKR3</latexit>

Process 3<latexit sha1_base64="M+DFRKJgeIxw8z9uzqdWjNQfvwE=">AAACInicbVDLSsNAFJ3UV42vqEsRBovgqiR1oe4KblxWsA9oSplMb9qhk0mYmYg1dOWvuHGrf+FOXAl+g9/gtM1C2x64cDjn3su9J0g4U9p1v6zCyura+kZx097a3tndc/YPGipOJYU6jXksWwFRwJmAumaaQyuRQKKAQzMYXk/85j1IxWJxp0cJdCLSFyxklGgjdZ1jn4LQIJno276GBx2EWU3GFJTC5+OuU3LL7hR4kXg5KaEcta7z4/dimkZmJ+VEqbbnJrqTEakZ5TC2/VRBQuiQ9KFtqCARqE42fWOMT43Sw2EsTQmNp+rfiYxESo2iwHRGRA/UvDcRl3oSuGKPsMxrpzq87GRMJKkGQWdHhCnHOsaTvHCPSaCajwwhVDLzB6YDIgk1oSnbBOTNx7FIGpWy55a920qpepVHVURH6ASdIQ9doCq6QTVURxQ9oRf0it6sZ+vd+rA+Z60FK585RP9gff8CUwKkeQ==</latexit><latexit sha1_base64="M+DFRKJgeIxw8z9uzqdWjNQfvwE=">AAACInicbVDLSsNAFJ3UV42vqEsRBovgqiR1oe4KblxWsA9oSplMb9qhk0mYmYg1dOWvuHGrf+FOXAl+g9/gtM1C2x64cDjn3su9J0g4U9p1v6zCyura+kZx097a3tndc/YPGipOJYU6jXksWwFRwJmAumaaQyuRQKKAQzMYXk/85j1IxWJxp0cJdCLSFyxklGgjdZ1jn4LQIJno276GBx2EWU3GFJTC5+OuU3LL7hR4kXg5KaEcta7z4/dimkZmJ+VEqbbnJrqTEakZ5TC2/VRBQuiQ9KFtqCARqE42fWOMT43Sw2EsTQmNp+rfiYxESo2iwHRGRA/UvDcRl3oSuGKPsMxrpzq87GRMJKkGQWdHhCnHOsaTvHCPSaCajwwhVDLzB6YDIgk1oSnbBOTNx7FIGpWy55a920qpepVHVURH6ASdIQ9doCq6QTVURxQ9oRf0it6sZ+vd+rA+Z60FK585RP9gff8CUwKkeQ==</latexit><latexit sha1_base64="M+DFRKJgeIxw8z9uzqdWjNQfvwE=">AAACInicbVDLSsNAFJ3UV42vqEsRBovgqiR1oe4KblxWsA9oSplMb9qhk0mYmYg1dOWvuHGrf+FOXAl+g9/gtM1C2x64cDjn3su9J0g4U9p1v6zCyura+kZx097a3tndc/YPGipOJYU6jXksWwFRwJmAumaaQyuRQKKAQzMYXk/85j1IxWJxp0cJdCLSFyxklGgjdZ1jn4LQIJno276GBx2EWU3GFJTC5+OuU3LL7hR4kXg5KaEcta7z4/dimkZmJ+VEqbbnJrqTEakZ5TC2/VRBQuiQ9KFtqCARqE42fWOMT43Sw2EsTQmNp+rfiYxESo2iwHRGRA/UvDcRl3oSuGKPsMxrpzq87GRMJKkGQWdHhCnHOsaTvHCPSaCajwwhVDLzB6YDIgk1oSnbBOTNx7FIGpWy55a920qpepVHVURH6ASdIQ9doCq6QTVURxQ9oRf0it6sZ+vd+rA+Z60FK585RP9gff8CUwKkeQ==</latexit><latexit sha1_base64="M+DFRKJgeIxw8z9uzqdWjNQfvwE=">AAACInicbVDLSsNAFJ3UV42vqEsRBovgqiR1oe4KblxWsA9oSplMb9qhk0mYmYg1dOWvuHGrf+FOXAl+g9/gtM1C2x64cDjn3su9J0g4U9p1v6zCyura+kZx097a3tndc/YPGipOJYU6jXksWwFRwJmAumaaQyuRQKKAQzMYXk/85j1IxWJxp0cJdCLSFyxklGgjdZ1jn4LQIJno276GBx2EWU3GFJTC5+OuU3LL7hR4kXg5KaEcta7z4/dimkZmJ+VEqbbnJrqTEakZ5TC2/VRBQuiQ9KFtqCARqE42fWOMT43Sw2EsTQmNp+rfiYxESo2iwHRGRA/UvDcRl3oSuGKPsMxrpzq87GRMJKkGQWdHhCnHOsaTvHCPSaCajwwhVDLzB6YDIgk1oSnbBOTNx7FIGpWy55a920qpepVHVURH6ASdIQ9doCq6QTVURxQ9oRf0it6sZ+vd+rA+Z60FK585RP9gff8CUwKkeQ==</latexit> Process 4

<latexit sha1_base64="qCiG5N8/wAn0jCb70h1ejyHuKuY=">AAACInicbVDLSsNAFJ3UV42vqEsRBovgqiRFUHcFNy4r2Ac0pUymN+3QySTMTMQauvJX3LjVv3AnrgS/wW9w2mahbQ9cOJxz7+XeEyScKe26X1ZhZXVtfaO4aW9t7+zuOfsHDRWnkkKdxjyWrYAo4ExAXTPNoZVIIFHAoRkMryd+8x6kYrG406MEOhHpCxYySrSRus6xT0FokEz0bV/Dgw7CrCZjCkrh83HXKblldwq8SLyclFCOWtf58XsxTSOzk3KiVNtzE93JiNSMchjbfqogIXRI+tA2VJAIVCebvjHGp0bp4TCWpoTGU/XvREYipUZRYDojogdq3puISz0JXLFHWOa1Ux1edjImklSDoLMjwpRjHeNJXrjHJFDNR4YQKpn5A9MBkYSa0JRtAvLm41gkjUrZc8vebaVUvcqjKqIjdILOkIcuUBXdoBqqI4qe0At6RW/Ws/VufVifs9aClc8con+wvn8BVKOkeg==</latexit><latexit sha1_base64="qCiG5N8/wAn0jCb70h1ejyHuKuY=">AAACInicbVDLSsNAFJ3UV42vqEsRBovgqiRFUHcFNy4r2Ac0pUymN+3QySTMTMQauvJX3LjVv3AnrgS/wW9w2mahbQ9cOJxz7+XeEyScKe26X1ZhZXVtfaO4aW9t7+zuOfsHDRWnkkKdxjyWrYAo4ExAXTPNoZVIIFHAoRkMryd+8x6kYrG406MEOhHpCxYySrSRus6xT0FokEz0bV/Dgw7CrCZjCkrh83HXKblldwq8SLyclFCOWtf58XsxTSOzk3KiVNtzE93JiNSMchjbfqogIXRI+tA2VJAIVCebvjHGp0bp4TCWpoTGU/XvREYipUZRYDojogdq3puISz0JXLFHWOa1Ux1edjImklSDoLMjwpRjHeNJXrjHJFDNR4YQKpn5A9MBkYSa0JRtAvLm41gkjUrZc8vebaVUvcqjKqIjdILOkIcuUBXdoBqqI4qe0At6RW/Ws/VufVifs9aClc8con+wvn8BVKOkeg==</latexit><latexit sha1_base64="qCiG5N8/wAn0jCb70h1ejyHuKuY=">AAACInicbVDLSsNAFJ3UV42vqEsRBovgqiRFUHcFNy4r2Ac0pUymN+3QySTMTMQauvJX3LjVv3AnrgS/wW9w2mahbQ9cOJxz7+XeEyScKe26X1ZhZXVtfaO4aW9t7+zuOfsHDRWnkkKdxjyWrYAo4ExAXTPNoZVIIFHAoRkMryd+8x6kYrG406MEOhHpCxYySrSRus6xT0FokEz0bV/Dgw7CrCZjCkrh83HXKblldwq8SLyclFCOWtf58XsxTSOzk3KiVNtzE93JiNSMchjbfqogIXRI+tA2VJAIVCebvjHGp0bp4TCWpoTGU/XvREYipUZRYDojogdq3puISz0JXLFHWOa1Ux1edjImklSDoLMjwpRjHeNJXrjHJFDNR4YQKpn5A9MBkYSa0JRtAvLm41gkjUrZc8vebaVUvcqjKqIjdILOkIcuUBXdoBqqI4qe0At6RW/Ws/VufVifs9aClc8con+wvn8BVKOkeg==</latexit><latexit sha1_base64="qCiG5N8/wAn0jCb70h1ejyHuKuY=">AAACInicbVDLSsNAFJ3UV42vqEsRBovgqiRFUHcFNy4r2Ac0pUymN+3QySTMTMQauvJX3LjVv3AnrgS/wW9w2mahbQ9cOJxz7+XeEyScKe26X1ZhZXVtfaO4aW9t7+zuOfsHDRWnkkKdxjyWrYAo4ExAXTPNoZVIIFHAoRkMryd+8x6kYrG406MEOhHpCxYySrSRus6xT0FokEz0bV/Dgw7CrCZjCkrh83HXKblldwq8SLyclFCOWtf58XsxTSOzk3KiVNtzE93JiNSMchjbfqogIXRI+tA2VJAIVCebvjHGp0bp4TCWpoTGU/XvREYipUZRYDojogdq3puISz0JXLFHWOa1Ux1edjImklSDoLMjwpRjHeNJXrjHJFDNR4YQKpn5A9MBkYSa0JRtAvLm41gkjUrZc8vebaVUvcqjKqIjdILOkIcuUBXdoBqqI4qe0At6RW/Ws/VufVifs9aClc8con+wvn8BVKOkeg==</latexit>

⌧ ⇠ 2Geo(↵)

y ⇠ Pois(µ)

y(±) := y + ⌧<latexit sha1_base64="pBV9WgJP/JfCuWpdQD4sOWUGEZQ=">AAACc3icbVFNaxsxENVu0jZ1P+K0x1xE3AaHUrMbAg2FQiCH9uhCnQQs18zK41hEWi3SbMh22X/ZS/9Eb7n0VHnjQ+tkQPB47w0zepMVWnlKkl9RvLH56PGTraedZ89fvNzu7rw687Z0EkfSausuMvCoVY4jUqTxonAIJtN4nl2dLvXza3Re2fwbVQVODFzmaq4kUKCm3VwQlHxfeGW4ILwhZ+rDz2ibvgBdLOCAC9Gp1gxDq3wwmLJVBSk9w7pqvtfCAC28Aa3R1X1RmIOm4fsfP/GKv+PLSdNuLxkkbfH7IF2BHlvVcNr9LWZWlgZzkhq8H6dJQZMaHCmpsemI0mMB8goucRxgDgb9pG5zafjbwMz43LrwcuIt+29HDcb7ymTB2S6+ri3JBzWH2qsf+JA2Lml+PKlVXpSEubxbYl5qTpYvD8BnyqEkXQUA0qnwDy4X4EBSOFMnBJSux3EfnB0O0mSQfj3qnRytotpiu2yP9VnKPrAT9oUN2YhJ9pP9iTaizeg23o334jd31jha9bxm/1X8/i+wR72V</latexit><latexit sha1_base64="pBV9WgJP/JfCuWpdQD4sOWUGEZQ=">AAACc3icbVFNaxsxENVu0jZ1P+K0x1xE3AaHUrMbAg2FQiCH9uhCnQQs18zK41hEWi3SbMh22X/ZS/9Eb7n0VHnjQ+tkQPB47w0zepMVWnlKkl9RvLH56PGTraedZ89fvNzu7rw687Z0EkfSausuMvCoVY4jUqTxonAIJtN4nl2dLvXza3Re2fwbVQVODFzmaq4kUKCm3VwQlHxfeGW4ILwhZ+rDz2ibvgBdLOCAC9Gp1gxDq3wwmLJVBSk9w7pqvtfCAC28Aa3R1X1RmIOm4fsfP/GKv+PLSdNuLxkkbfH7IF2BHlvVcNr9LWZWlgZzkhq8H6dJQZMaHCmpsemI0mMB8goucRxgDgb9pG5zafjbwMz43LrwcuIt+29HDcb7ymTB2S6+ri3JBzWH2qsf+JA2Lml+PKlVXpSEubxbYl5qTpYvD8BnyqEkXQUA0qnwDy4X4EBSOFMnBJSux3EfnB0O0mSQfj3qnRytotpiu2yP9VnKPrAT9oUN2YhJ9pP9iTaizeg23o334jd31jha9bxm/1X8/i+wR72V</latexit><latexit sha1_base64="pBV9WgJP/JfCuWpdQD4sOWUGEZQ=">AAACc3icbVFNaxsxENVu0jZ1P+K0x1xE3AaHUrMbAg2FQiCH9uhCnQQs18zK41hEWi3SbMh22X/ZS/9Eb7n0VHnjQ+tkQPB47w0zepMVWnlKkl9RvLH56PGTraedZ89fvNzu7rw687Z0EkfSausuMvCoVY4jUqTxonAIJtN4nl2dLvXza3Re2fwbVQVODFzmaq4kUKCm3VwQlHxfeGW4ILwhZ+rDz2ibvgBdLOCAC9Gp1gxDq3wwmLJVBSk9w7pqvtfCAC28Aa3R1X1RmIOm4fsfP/GKv+PLSdNuLxkkbfH7IF2BHlvVcNr9LWZWlgZzkhq8H6dJQZMaHCmpsemI0mMB8goucRxgDgb9pG5zafjbwMz43LrwcuIt+29HDcb7ymTB2S6+ri3JBzWH2qsf+JA2Lml+PKlVXpSEubxbYl5qTpYvD8BnyqEkXQUA0qnwDy4X4EBSOFMnBJSux3EfnB0O0mSQfj3qnRytotpiu2yP9VnKPrAT9oUN2YhJ9pP9iTaizeg23o334jd31jha9bxm/1X8/i+wR72V</latexit><latexit sha1_base64="pBV9WgJP/JfCuWpdQD4sOWUGEZQ=">AAACc3icbVFNaxsxENVu0jZ1P+K0x1xE3AaHUrMbAg2FQiCH9uhCnQQs18zK41hEWi3SbMh22X/ZS/9Eb7n0VHnjQ+tkQPB47w0zepMVWnlKkl9RvLH56PGTraedZ89fvNzu7rw687Z0EkfSausuMvCoVY4jUqTxonAIJtN4nl2dLvXza3Re2fwbVQVODFzmaq4kUKCm3VwQlHxfeGW4ILwhZ+rDz2ibvgBdLOCAC9Gp1gxDq3wwmLJVBSk9w7pqvtfCAC28Aa3R1X1RmIOm4fsfP/GKv+PLSdNuLxkkbfH7IF2BHlvVcNr9LWZWlgZzkhq8H6dJQZMaHCmpsemI0mMB8goucRxgDgb9pG5zafjbwMz43LrwcuIt+29HDcb7ymTB2S6+ri3JBzWH2qsf+JA2Lml+PKlVXpSEubxbYl5qTpYvD8BnyqEkXQUA0qnwDy4X4EBSOFMnBJSux3EfnB0O0mSQfj3qnRytotpiu2yP9VnKPrAT9oUN2YhJ9pP9iTaizeg23o334jd31jha9bxm/1X8/i+wR72V</latexit>

�(⇤) ⇠ Exp( ↵1�↵ )

g(⇤) ⇠ Pois(�(⇤))

⌧ := g(+) � g(�)

y ⇠ Pois(µ)

y(±) := y + ⌧<latexit sha1_base64="Eg59HZOJ+lbkOiY3p4FrRZnuJlA=">AAADIHicjVJNi9RAEO3Er93xY2f1ppfGwWXGZYZEFhRBWBDB4wjO7sL0OHQ6lZlmu5PQ3ZGNTQ7+lb3sVf/F3sSj/gh/g51sDjozggWBx6uq96orFeWCaxMEPzz/2vUbN29tbXdu37l7b6e7e/9IZ4ViMGGZyNRJRDUInsLEcCPgJFdAZSTgODp9XeePP4LSPEvfmzKHmaSLlCecUeOo+a73kAhXHdMPlkhqllpSIUDZ/tNBVeE9ornExMCZUdK+OcurPjGJoswSKvIlrWyIh7jFA0I6i/+QGWdcO51/29ZCxNAC7718hdcU92vF4To/rHnXWW62k8UAN7pcxGDLarWd5LIZ1VmWeB/X/vNuLxgFTeB1ELagh9oYz7u/SJyxQkJqmKBaT8MgNzNLleFMQNUhhYacslO6gKmDKZWgZ7b5iRV+4pgYJ5lyX2pww/7ZYanUupSRq2wGX83V5MacAqH5J9iUmxYmeTGzPM0LAym7GiIpBDYZrq8Fx1wBM6J0gDLF3TswW1J3AMbdVMctKFxdxzo4ejYKg1H47qB3eNCuags9Qo9RH4XoOTpEb9EYTRDzPnsX3hfvq3/uX/rf/O9Xpb7X9jxAf4X/8zdi8f/C</latexit><latexit sha1_base64="Eg59HZOJ+lbkOiY3p4FrRZnuJlA=">AAADIHicjVJNi9RAEO3Er93xY2f1ppfGwWXGZYZEFhRBWBDB4wjO7sL0OHQ6lZlmu5PQ3ZGNTQ7+lb3sVf/F3sSj/gh/g51sDjozggWBx6uq96orFeWCaxMEPzz/2vUbN29tbXdu37l7b6e7e/9IZ4ViMGGZyNRJRDUInsLEcCPgJFdAZSTgODp9XeePP4LSPEvfmzKHmaSLlCecUeOo+a73kAhXHdMPlkhqllpSIUDZ/tNBVeE9ornExMCZUdK+OcurPjGJoswSKvIlrWyIh7jFA0I6i/+QGWdcO51/29ZCxNAC7718hdcU92vF4To/rHnXWW62k8UAN7pcxGDLarWd5LIZ1VmWeB/X/vNuLxgFTeB1ELagh9oYz7u/SJyxQkJqmKBaT8MgNzNLleFMQNUhhYacslO6gKmDKZWgZ7b5iRV+4pgYJ5lyX2pww/7ZYanUupSRq2wGX83V5MacAqH5J9iUmxYmeTGzPM0LAym7GiIpBDYZrq8Fx1wBM6J0gDLF3TswW1J3AMbdVMctKFxdxzo4ejYKg1H47qB3eNCuags9Qo9RH4XoOTpEb9EYTRDzPnsX3hfvq3/uX/rf/O9Xpb7X9jxAf4X/8zdi8f/C</latexit><latexit sha1_base64="Eg59HZOJ+lbkOiY3p4FrRZnuJlA=">AAADIHicjVJNi9RAEO3Er93xY2f1ppfGwWXGZYZEFhRBWBDB4wjO7sL0OHQ6lZlmu5PQ3ZGNTQ7+lb3sVf/F3sSj/gh/g51sDjozggWBx6uq96orFeWCaxMEPzz/2vUbN29tbXdu37l7b6e7e/9IZ4ViMGGZyNRJRDUInsLEcCPgJFdAZSTgODp9XeePP4LSPEvfmzKHmaSLlCecUeOo+a73kAhXHdMPlkhqllpSIUDZ/tNBVeE9ornExMCZUdK+OcurPjGJoswSKvIlrWyIh7jFA0I6i/+QGWdcO51/29ZCxNAC7718hdcU92vF4To/rHnXWW62k8UAN7pcxGDLarWd5LIZ1VmWeB/X/vNuLxgFTeB1ELagh9oYz7u/SJyxQkJqmKBaT8MgNzNLleFMQNUhhYacslO6gKmDKZWgZ7b5iRV+4pgYJ5lyX2pww/7ZYanUupSRq2wGX83V5MacAqH5J9iUmxYmeTGzPM0LAym7GiIpBDYZrq8Fx1wBM6J0gDLF3TswW1J3AMbdVMctKFxdxzo4ejYKg1H47qB3eNCuags9Qo9RH4XoOTpEb9EYTRDzPnsX3hfvq3/uX/rf/O9Xpb7X9jxAf4X/8zdi8f/C</latexit><latexit sha1_base64="Eg59HZOJ+lbkOiY3p4FrRZnuJlA=">AAADIHicjVJNi9RAEO3Er93xY2f1ppfGwWXGZYZEFhRBWBDB4wjO7sL0OHQ6lZlmu5PQ3ZGNTQ7+lb3sVf/F3sSj/gh/g51sDjozggWBx6uq96orFeWCaxMEPzz/2vUbN29tbXdu37l7b6e7e/9IZ4ViMGGZyNRJRDUInsLEcCPgJFdAZSTgODp9XeePP4LSPEvfmzKHmaSLlCecUeOo+a73kAhXHdMPlkhqllpSIUDZ/tNBVeE9ornExMCZUdK+OcurPjGJoswSKvIlrWyIh7jFA0I6i/+QGWdcO51/29ZCxNAC7718hdcU92vF4To/rHnXWW62k8UAN7pcxGDLarWd5LIZ1VmWeB/X/vNuLxgFTeB1ELagh9oYz7u/SJyxQkJqmKBaT8MgNzNLleFMQNUhhYacslO6gKmDKZWgZ7b5iRV+4pgYJ5lyX2pww/7ZYanUupSRq2wGX83V5MacAqH5J9iUmxYmeTGzPM0LAym7GiIpBDYZrq8Fx1wBM6J0gDLF3TswW1J3AMbdVMctKFxdxzo4ejYKg1H47qB3eNCuags9Qo9RH4XoOTpEb9EYTRDzPnsX3hfvq3/uX/rf/O9Xpb7X9jxAf4X/8zdi8f/C</latexit>

�(⇤) ⇠ Exp( ↵1�↵ )

g(�) ⇠ Pois(�(�))

y(+) ⇠ Pois(µ+�(+))

y(±) := y(+) � g(�)

<latexit sha1_base64="nmumql7nYhfXtkXrJMCFqkA+jpc=">AAADTnichVJLa9tAEF4pfbjuy0mPvSw1LU6NjVQCDYFAoBRydKFOAl7XrFYje8muJHZXIc6i39S/0ksPvbT/oMfeSrtSdGhtpx1Y+Jj55pvHTpQLrk0QfPH8rVu379xt3Wvff/Dw0ePO9s6JzgrFYMwykamziGoQPIWx4UbAWa6AykjAaXT+poqfXoDSPEvfm2UOU0nnKU84o8a5ZtveMRGOHdMPlkhqFlpSIUDZ3svdssQviOYSEwOXRkn79jIve8QkijJLqMgXtLQhHuAG7xLSnq/KDNZlRhnXTmdz2YpfCRHDRQx2Wa4S+jcKygLbfolvEO7/T5jkspY+OMT/Lj7AG6ecdbrBMKgNr4OwAV3U2GjW+U7ijBUSUsME1XoSBrmZWqoMZwLKNik05JSd0zlMHEypBD219ZeX+LnzxDjJlHupwbX3zwxLpdZLGTlm3elqrHJujCkQml/BptikMMn+1PI0Lwyk7LqJpBDYZLi6LRxzBcyIpQOUKe7mwGxB3bkYd4Ftt6BwdR3r4OTVMAyG4bu97tFes6oWeoqeoR4K0Wt0hI7RCI0R8z56n72v3jf/k//D/+n/uqb6XpPzBP1lW63fHPUVEw==</latexit><latexit sha1_base64="nmumql7nYhfXtkXrJMCFqkA+jpc=">AAADTnichVJLa9tAEF4pfbjuy0mPvSw1LU6NjVQCDYFAoBRydKFOAl7XrFYje8muJHZXIc6i39S/0ksPvbT/oMfeSrtSdGhtpx1Y+Jj55pvHTpQLrk0QfPH8rVu379xt3Wvff/Dw0ePO9s6JzgrFYMwykamziGoQPIWx4UbAWa6AykjAaXT+poqfXoDSPEvfm2UOU0nnKU84o8a5ZtveMRGOHdMPlkhqFlpSIUDZ3svdssQviOYSEwOXRkn79jIve8QkijJLqMgXtLQhHuAG7xLSnq/KDNZlRhnXTmdz2YpfCRHDRQx2Wa4S+jcKygLbfolvEO7/T5jkspY+OMT/Lj7AG6ecdbrBMKgNr4OwAV3U2GjW+U7ijBUSUsME1XoSBrmZWqoMZwLKNik05JSd0zlMHEypBD219ZeX+LnzxDjJlHupwbX3zwxLpdZLGTlm3elqrHJujCkQml/BptikMMn+1PI0Lwyk7LqJpBDYZLi6LRxzBcyIpQOUKe7mwGxB3bkYd4Ftt6BwdR3r4OTVMAyG4bu97tFes6oWeoqeoR4K0Wt0hI7RCI0R8z56n72v3jf/k//D/+n/uqb6XpPzBP1lW63fHPUVEw==</latexit><latexit sha1_base64="nmumql7nYhfXtkXrJMCFqkA+jpc=">AAADTnichVJLa9tAEF4pfbjuy0mPvSw1LU6NjVQCDYFAoBRydKFOAl7XrFYje8muJHZXIc6i39S/0ksPvbT/oMfeSrtSdGhtpx1Y+Jj55pvHTpQLrk0QfPH8rVu379xt3Wvff/Dw0ePO9s6JzgrFYMwykamziGoQPIWx4UbAWa6AykjAaXT+poqfXoDSPEvfm2UOU0nnKU84o8a5ZtveMRGOHdMPlkhqFlpSIUDZ3svdssQviOYSEwOXRkn79jIve8QkijJLqMgXtLQhHuAG7xLSnq/KDNZlRhnXTmdz2YpfCRHDRQx2Wa4S+jcKygLbfolvEO7/T5jkspY+OMT/Lj7AG6ecdbrBMKgNr4OwAV3U2GjW+U7ijBUSUsME1XoSBrmZWqoMZwLKNik05JSd0zlMHEypBD219ZeX+LnzxDjJlHupwbX3zwxLpdZLGTlm3elqrHJujCkQml/BptikMMn+1PI0Lwyk7LqJpBDYZLi6LRxzBcyIpQOUKe7mwGxB3bkYd4Ftt6BwdR3r4OTVMAyG4bu97tFes6oWeoqeoR4K0Wt0hI7RCI0R8z56n72v3jf/k//D/+n/uqb6XpPzBP1lW63fHPUVEw==</latexit><latexit sha1_base64="nmumql7nYhfXtkXrJMCFqkA+jpc=">AAADTnichVJLa9tAEF4pfbjuy0mPvSw1LU6NjVQCDYFAoBRydKFOAl7XrFYje8muJHZXIc6i39S/0ksPvbT/oMfeSrtSdGhtpx1Y+Jj55pvHTpQLrk0QfPH8rVu379xt3Wvff/Dw0ePO9s6JzgrFYMwykamziGoQPIWx4UbAWa6AykjAaXT+poqfXoDSPEvfm2UOU0nnKU84o8a5ZtveMRGOHdMPlkhqFlpSIUDZ3svdssQviOYSEwOXRkn79jIve8QkijJLqMgXtLQhHuAG7xLSnq/KDNZlRhnXTmdz2YpfCRHDRQx2Wa4S+jcKygLbfolvEO7/T5jkspY+OMT/Lj7AG6ecdbrBMKgNr4OwAV3U2GjW+U7ijBUSUsME1XoSBrmZWqoMZwLKNik05JSd0zlMHEypBD219ZeX+LnzxDjJlHupwbX3zwxLpdZLGTlm3elqrHJujCkQml/BptikMMn+1PI0Lwyk7LqJpBDYZLi6LRxzBcyIpQOUKe7mwGxB3bkYd4Ftt6BwdR3r4OTVMAyG4bu97tFes6oWeoqeoR4K0Wt0hI7RCI0R8z56n72v3jf/k//D/+n/uqb6XpPzBP1lW63fHPUVEw==</latexit>

�(⇤) ⇠ Exp( ↵1�↵ )

y(±) ⇠ Skel��(+)+µ, �(�)

�<latexit sha1_base64="idxw1tNbXhiZuiNvUTkk3kTnkCc=">AAAC03icdVJNbxMxEPUuXyV8NIUjF5cIlNA22kWR4FgJIXEsgrQVcYi83tnEir27smerBmsviCt/jTt/gt+AN90DJO1Ilp7eezMaPzsplbQYRb+D8NbtO3fv7dzvPHj46PFud+/JqS0qI2AsClWY84RbUDKHMUpUcF4a4DpRcJYs3zX62QUYK4v8M65KmGo+z2UmBUdPzbq/mPLmlH91THNcWM2VAuP6rwZ1TV8yKzVlCJdotHt/WdZ9hpnhwjGuygWvXUyPaIsHjHUYSpWCW9Wb41iptwd+WoKqWSLn/RuWOGh62P4B26dMV4fskN5gPPLGZtBg1u1Fw2hddBvELeiRtk5m3T8sLUSlIUehuLWTOCpx6rhBKRTUHVZZKLlY8jlMPMy5Bjt169xr+sIzKc0K40+OdM3+2+G4tnalE+9cr7upNeS1mgFl5Te4TptUmL2dOpmXFUIurpbIKkWxoM0D01QaEKhWHnBhpL8HFQvuHw39N+j4gOLNOLbB6ethHA3jj6Pe8aiNaoc8I89Jn8TkDTkmH8gJGRMRjIIvgQjScBy68Hv448oaBm3PU/JfhT//AiJ7480=</latexit><latexit sha1_base64="idxw1tNbXhiZuiNvUTkk3kTnkCc=">AAAC03icdVJNbxMxEPUuXyV8NIUjF5cIlNA22kWR4FgJIXEsgrQVcYi83tnEir27smerBmsviCt/jTt/gt+AN90DJO1Ilp7eezMaPzsplbQYRb+D8NbtO3fv7dzvPHj46PFud+/JqS0qI2AsClWY84RbUDKHMUpUcF4a4DpRcJYs3zX62QUYK4v8M65KmGo+z2UmBUdPzbq/mPLmlH91THNcWM2VAuP6rwZ1TV8yKzVlCJdotHt/WdZ9hpnhwjGuygWvXUyPaIsHjHUYSpWCW9Wb41iptwd+WoKqWSLn/RuWOGh62P4B26dMV4fskN5gPPLGZtBg1u1Fw2hddBvELeiRtk5m3T8sLUSlIUehuLWTOCpx6rhBKRTUHVZZKLlY8jlMPMy5Bjt169xr+sIzKc0K40+OdM3+2+G4tnalE+9cr7upNeS1mgFl5Te4TptUmL2dOpmXFUIurpbIKkWxoM0D01QaEKhWHnBhpL8HFQvuHw39N+j4gOLNOLbB6ethHA3jj6Pe8aiNaoc8I89Jn8TkDTkmH8gJGRMRjIIvgQjScBy68Hv448oaBm3PU/JfhT//AiJ7480=</latexit><latexit sha1_base64="idxw1tNbXhiZuiNvUTkk3kTnkCc=">AAAC03icdVJNbxMxEPUuXyV8NIUjF5cIlNA22kWR4FgJIXEsgrQVcYi83tnEir27smerBmsviCt/jTt/gt+AN90DJO1Ilp7eezMaPzsplbQYRb+D8NbtO3fv7dzvPHj46PFud+/JqS0qI2AsClWY84RbUDKHMUpUcF4a4DpRcJYs3zX62QUYK4v8M65KmGo+z2UmBUdPzbq/mPLmlH91THNcWM2VAuP6rwZ1TV8yKzVlCJdotHt/WdZ9hpnhwjGuygWvXUyPaIsHjHUYSpWCW9Wb41iptwd+WoKqWSLn/RuWOGh62P4B26dMV4fskN5gPPLGZtBg1u1Fw2hddBvELeiRtk5m3T8sLUSlIUehuLWTOCpx6rhBKRTUHVZZKLlY8jlMPMy5Bjt169xr+sIzKc0K40+OdM3+2+G4tnalE+9cr7upNeS1mgFl5Te4TptUmL2dOpmXFUIurpbIKkWxoM0D01QaEKhWHnBhpL8HFQvuHw39N+j4gOLNOLbB6ethHA3jj6Pe8aiNaoc8I89Jn8TkDTkmH8gJGRMRjIIvgQjScBy68Hv448oaBm3PU/JfhT//AiJ7480=</latexit><latexit sha1_base64="idxw1tNbXhiZuiNvUTkk3kTnkCc=">AAAC03icdVJNbxMxEPUuXyV8NIUjF5cIlNA22kWR4FgJIXEsgrQVcYi83tnEir27smerBmsviCt/jTt/gt+AN90DJO1Ilp7eezMaPzsplbQYRb+D8NbtO3fv7dzvPHj46PFud+/JqS0qI2AsClWY84RbUDKHMUpUcF4a4DpRcJYs3zX62QUYK4v8M65KmGo+z2UmBUdPzbq/mPLmlH91THNcWM2VAuP6rwZ1TV8yKzVlCJdotHt/WdZ9hpnhwjGuygWvXUyPaIsHjHUYSpWCW9Wb41iptwd+WoKqWSLn/RuWOGh62P4B26dMV4fskN5gPPLGZtBg1u1Fw2hddBvELeiRtk5m3T8sLUSlIUehuLWTOCpx6rhBKRTUHVZZKLlY8jlMPMy5Bjt169xr+sIzKc0K40+OdM3+2+G4tnalE+9cr7upNeS1mgFl5Te4TptUmL2dOpmXFUIurpbIKkWxoM0D01QaEKhWHnBhpL8HFQvuHw39N+j4gOLNOLbB6ethHA3jj6Pe8aiNaoc8I89Jn8TkDTkmH8gJGRMRjIIvgQjScBy68Hv448oaBm3PU/JfhT//AiJ7480=</latexit>

Figure 2. Four generative processes that yield the same marginal distribution P (y(±) |µ, α).

4. MCMC algorithmGiven a privatized data set Y (±), our inference goal is toapproximate the locally private posterior using MCMC. Todo this, we need to be able to sample values of the non-privatized data set Y from its complete conditional, as ex-plained in section 2. By assuming that each privatized county(±)

i is a Skellam random variable, as in equation 11, we canexploit the following general theorem that relates the Skel-lam and Bessel (Yuan & Kalbfleisch, 2000) distributions.Theorem 3. (Proof in appendix) Consider two Poissonrandom variables y1 ∼ Pois(λ(+)) and y2 ∼ Pois(λ(−)).Their minimum m := min{y1, y2} and their differenceτ := y1 − y2 are deterministic functions of y1 and y2.However, if not conditioned on y1 and y2, the randomvariables m and τ can be marginally generated as follows:

τ ∼ Skel(λ(+), λ(−)), m ∼ Bes(|τ |, 2

√λ(+)λ(−)

). (12)

Theorem 3 means that we can generate two independentPoisson random variables by first generating their differenceτ and then their minimum m. Because τ = y1 − y2, if τ ispositive, then y2 must be the minimum and thus y1 = τ+m.In practice, this means that if we only get to observe thedifference of two Poisson-distributed counts, we can still“recover” the counts by sampling a Bessel auxiliary variable.

By assuming that y(±)

i ∼ Skel(λ(+)

i +µi, λ(−)

i ) via theorem 2,we can sample an auxiliary Bessel random variable mi:

(mi | −

)∼ Bes

(|y(±)

i |, 2√

(λ(+)

i +µi)λ(−)

i

). (13)

Yuan & Kalbfleisch (2000) give details of the Bessel distri-bution, which can be sampled efficiently (Devroye, 2002).

Via theorem 3, mi represents the minimum of two latentPoisson random variables whose difference equals y(±)

i ;these latent variables are depicted in process 3 of figure 2—i.e., y(±)

i := y(+)

i − g(−)

i and mi = min{y(+)

i , g(−)

i }. Giveny(±)

i and a sampled value of mi, we can compute y(+)

i , g(−)

i :

y(+)

i := mi, g(−)

i := y(+)

i − y(±)

i if y(±)

i ≤ 0 (14)

g(−)

i := mi, y(+)

i := g(−)

i + y(±)

i otherwise.

Because y(+)

i =yi+g(+)

i is itself the sum of two independentPoisson random variables, we can then sample yi from itsconditional posterior, which is a binomial distribution:

(yi | −

)∼ Binom

(y(+)

i , µi

µi+λ(+)i

). (15)

Equations 13 through 15 constitute a way to draw samplesfrom PM,R(yi |µi, y

(±)

i ,λi). We can also sample the aux-iliary variables λ(+)

i , λ(−)

i from their complete conditional:

(λ(∗)i | −

)∼ Γ

(1 + g(∗)

i , α1−α + 1

). (16)

Iteratively re-sampling yi and λi constitutes a chain whosestationary distribution over yi is PM,R(yi |µi, y

(±)

i ), asdesired. Given a sampled non-privatized data set Y , wecan then sample values of the latent variables Z from theircomplete conditionals, which are the same as in non-privatePoisson factorization. Equations 13–16, along with thecomplete conditionals for Z, define an efficient MCMCalgorithm that is asymptotically guaranteed to generatesamples from the locally private posterior PM,R(Z | Y (±)).


word type v

docu

men

td

True topics �⇤k

Non-private estimate �k

word type v

(a) No noise.

word type v

docu

men

td

word type v

Our estimate �k

Naıve estimate �k

(b) Low noise.

word type v

docu

men

td

word type v


Our estimate �k


(c) High noise.

Figure 3. Topic recovery: our method vs. the naıve approach. (a) We generated the non-privatized data set synthetically so that the truetopics were known. We then privatized the data set using (b) a low noise level and (c) a high noise level. The heatmap in each subfigurevisualizes the data set, using red to denote positive counts and blue to denote negative counts. With a high level of noise, the naıveapproach overfits the noise and therefore fails to recover the true topics. In contrast, our method is still able to recover the true topics.

5. Case studiesWe present two case studies applying our method to 1) topicmodeling for text corpora and 2) overlapping communitydetection for social networks. In each one, we formulate pri-vacy guarantees, ground them in examples, and demonstrateour method’s utility using synthetic and real-world data.

Real-world data. For our experiments using real-worlddata, we derived count matrices from the Enron emailcorpus (Klimt & Yang, 2004). For the topic modelingcase study, we randomly selected D=10, 000 emails withat least 50 word tokens. We limited the vocabulary toV = 10, 000 word types by selecting the most frequentword types with document frequencies less than 0.3. For thecommunity detection case study, we obtained a V ×V adja-cency matrix Y where yij is the number of emails sent fromactor i to actor j. We included an actor if they sent at leastone email and sent or received at least one hundred emails,yielding V =161 actors. When an email included multiplerecipients, we incremented the corresponding counts by one.

Reference methods. We compare the performance of ourmethod to two references methods: 1) non-private Poissonfactorization of the non-privatized data and 2) the naıveapproach, wherein inference treats the privatized data as if itwere not privatized. The naıve approach must first truncateany negative counts y(±)

i <0 to zero and thus implicitly usesthe truncated geometric mechanism (Ghosh et al., 2012).

Performance measures. Each method uses MCMC to ap-proximate the posterior with a set of S samples of the latentvariables. We can therefore use these samples to approxi-mate each method’s posterior expectation of µi as follows:

µi =1

S

S∑

s=1

µ(s)i ≈ EPM,R(Z | Y (±)) [µi] . (17)

We can then calculate the mean absolute error (MAE) of

µi with respect to yi—i.e., the reconstruction error. In thetopic modeling case study, we also compare the quality ofthe inferred topics using two standard metrics: 1) normal-ized pointwise mutual information (NPMI; Lau et al., 2014)and 2) coherence (Mimno et al., 2011). We use the non-privatized data set as the reference corpus for both. For eachmetric, we use only the top ten word types for each topic,and we average the topics’ scores to yield a single value.

5.1. Case study 1: Topic modeling

Topic models (Blei et al., 2003) are widely used in thesocial sciences (Ramage et al., 2009; Grimmer & Stewart,2013; Mohr & Bogdanov, 2013; Roberts et al., 2013) tocharacterize the high-level thematic structure of text corporavia latent “topics”—i.e., probability distributions over theword types in some vocabulary. However, in many scenarios,documents can contain sensitive information, so people maybe unwilling to share them without privacy guarantees.

Limited-precision local privacy. In the context of topicmodeling, a data set Y is a D×V count matrix, whereeach element ydv ∈ Z+ represents the number of timesthat word type v ∈ [V ] occurs in document d ∈ [D]. It isnatural to consider each document yd≡ (yd1, . . . , ydV ) tobe a single observation. Under LPLP, N determines theneighborhood of documents within which ε-local privacyholds. For example, if N = 4, then a document in whicha word type occurs four times and an otherwise-identicaldocument in which it does not occur at all would be renderedindistinguishable after privatization, assuming ε is small.

Poisson factorization. Latent Dirichlet allocation (Bleiet al., 2003), the most commonly used topic model, can bethought of as a special case of Poisson factorization whereY is a D×V count matrix and µdv =

∑Kk=1 θdk φkv. The

factor θdk represents how much topic k is used in document


123ε/N = ln(1/α)

0.1

0.2

0.3

0.4

Rec

onst

ruct

ion

MA

E

Model Type

Naıve

Proposed

Non-private

123ε/N = ln(1/α)

1.0

1.5

2.0

2.5

3.0

NP

MI

123ε/N = ln(1/α)

-80.0

-75.0

-70.0

-65.0

-60.0

Coh

eren

ce

Figure 4. Each subplot compares our method to the two reference methods for increasing levels of privacy. The left subplot depictsreconstruction error (lower values are better), the center subplot depicts NPMI (higher values are better), and the right subplot depictscoherence (higher values are better). Our method has lower reconstruction error and higher quality topics than the naıve approach.

d, while the factor φkv represents how much word typev is used in topic k. The set of latent variables is thusZ = {Θ,Φ}, where Θ and Φ are D×K and K×V non-negative, real-valued matrices, respectively. It is standardto assume independent gamma priors over the factors—i.e.,θdk, φkv ∼ Γ(a0, b0), where a0 and b0 are shape and ratehyperparameters respectively. We set a0 =0.1 and b0 =1.

Experiments using synthetic data. We generated a syn-thetic data set of D = 90 documents, with K = 3 topicsand V = 15 word types. We set Φ∗ so that the topicswere well separated, with each putting the majority of itsmass on five different word types. We also ensured thatthe documents were well separated into three equal groupsof thirty, with each document putting the majority of itsmass on a different topic. We then sampled a data sety∗dv ∼ Pois(µ∗dv) where µ∗dv =

∑Kk=1 θ

∗dkφ∗kv. We then

generated a heterogeneously-noised data set by samplingthe dth document’s noise level αd ∼ Beta

(c α0, c (1−α0)

)

from a Beta distribution with mean α0 and concentrationparameter c = 10 and then sampling τdv ∼ 2Geo(αd) foreach word type v. We repeated this for a small and largevalue of α0. For each method and data set, we ran 6,000MCMC iterations, saving every 25th sample after the first1,000. We selected Φ to be the sample from the posteriorwith the highest joint probability. (Due to label switching,we could not average samples of Φ.) Following Newmanet al. (2009), we then aligned the topic indices of Φ to Φ∗

using the Hungarian bipartite matching algorithm. We visu-alize the results in figure 3. The naıve approach performspoorly at recovering the topics under high levels of privacy.

Experiments using real-world data. We used three pri-vacy levels ε/N ∈ {3, 2, 1}. We generated five privatizeddata sets for each privacy level by adding noise drawn froma two-sided geometric distribution with α = − exp(ε/N)independently to each element of the non-privatized dataset. We applied our method and naıve approach to each ofthe fifteen privatized data sets and applied non-privatizedPoisson factorization to the non-privatized data set. Foreach method and data set, we used K = 50 topics and ran7,500 iterations of MCMC, saving every 100th sample of

the latent variables after the first 2,500. We used the fiftysaved samples to compute µdv = 1

S

∑Ss=1

∑Kk=1 θ

(s)dk φ

(s)kv .

We also computed NPMI and coherence using each savedsample, and averaged the resulting values over the samples.

Results. We find that our method has both lower recon-struction error and higher quality topics than the naıveapproach. The reconstruction error for each method anddata set is shown in left subplot of figure 4. Our methodhas almost the same reconstruction error as non-privatePoisson factorization of the non-privatized data. In contrast,the naıve approach has a higher reconstruction errorthat increases dramatically as the privacy level increases.The center and right subplots of figure 4 depict NPMIand coherence, respectively. According to both metrics,our method yields higher quality topics than the naıveapproach. Surprisingly, the topics inferred by our methodhave better coherence than the topics inferred by Poissonfactorization of the non-privatized data. This result suggeststhat non-private Poisson factorization may be overfitting;in contrast, our method may avoid overfitting by attributingsmall counts to the added noise and ignoring them. BecauseNPMI places more emphasis on rarer word types, excludingless reliable rare word types from topics does not benefitthe metric, as is reflected in the center subplot of figure 4.

5.2. Case study 2: Overlapping community detection

Organizations often want to know whether their employeesare interacting as productively as possible. For example, arethere missing links between their employees that, if present,would reduce duplication of effort? Do the “communities”that emerge naturally from employee interactions match theformal organizational structure? Although social scientistsmay be able gain answer such questions using employee in-teraction data, sharing such data increases the risk of privacyviolations. Moreover, standard anonymization procedurescan be reverse-engineered adversarially and thus do not pro-vide privacy guarantees (Narayanan & Shmatikov, 2009).

Limited-precision local privacy. In this context, a data setY is a V ×V count matrix, where each element yij ∈ Z+


123

ε/N = ln(1/α)

0.6

0.7

0.8

0.9

1.0

1.1

1.2

1.3

Rec

onst

ruct

ion

MA

ENumber of communities = 5

Model Type

Naıve

Proposed

Non-private

123

ε/N = ln(1/α)

Number of communities = 20

123

ε/N = ln(1/α)

7.0

8.0

9.0

10.0

11.0

12.0

Hel

d-ou

tM

AE


Model Type

Naıve

Proposed

Non-private

123

ε/N = ln(1/α)


Figure 5. Each subplot compares our method to the two reference methods for increasing levels of privacy. The left subplot and the centerleft subplot depict reconstruction error (lower values are better) for C = 5 and C = 20; the center right subplot and the right subplotdepict held-out reconstruction error (lower values are better) for C = 5 and C = 20. We provide results for C = 10 in the appendix.

represents the number of interactions from actor i ∈ [V ]to actor j ∈ [V ]. It is natural to consider each count yijto be a single observation. Under LPLP, if yij ≤ N , thenits privatized version will be indistinguishable from theprivatized version of yij = 0. In other words, an adversarywill be unable to tell from the privatized count whether ihad interacted with j at all, assuming ε is small. If yij � N ,then only the exact number of interactions will be concealed.

Poisson factorization model. The mixed-membershipstochastic block model (Ball et al., 2011; Gopalan & Blei,2013; Zhou, 2015) is a special case of Poisson factor-ization where Y is a V × V count matrix and µij =∑Cc=1

∑Cd=1 θic θjd πcd. The factors θic and θjd represent

how much actors i and j participate in communities c andd, respectively, while the factor πcd represents how muchactors in community c interact with actors in community d.The set of latent variables is thus Z={Θ,Π} where Θ andΠ are V×C andC×C non-negative, real-valued matrices, re-spectively. As with latent Dirichlet allocation, it is standardto assume independent gamma priors over the factors—i.e.,θic, πcd ∼ Γ(a0, b0), where a0 and b0 are shape and ratehyperparameters, respectively. We set a0 =0.1 and b0 =1.

Experiments using synthetic data. We generated socialnetworks of V = 20 actors with C = 5 communities. Werandomly generated the true parameters θ∗ic, π

∗cd∼Γ(a0, b0)

with a0 = 0.01 and b0 = 0.5 to encourage sparsity; doingso exaggerates the block structure in the network. Wethen sampled a data set yij ∼ Pois(µ∗ij) and added noiseτij ∼ 2Geo(α) for three increasing values of α. For eachdata set, we set N = E[yij ] and then set α = exp(−ε/N)for ε ∈ {2.5, 1, 0.75}. For each method and data set, weran 8,500 MCMC iterations, saving every 25th sample afterthe first 1,000 and using these samples to compute µij . Infigure 1, we visually compare our method’s estimates of µijto those of the reference methods. The naıve approach over-fits the noise, predicting high values even for sparse partsof the matrix. In contrast, our method approach reproducesthe sparse block structure even for high levels of privacy.

Experiments using real-world data. We used the sameexperimental design that we used in the topic modelingcase study. For each method and data set, we used C ∈{5, 10, 20} and we used the saved samples to computeµij = 1

S

∑Ss=1

∑Cc=1

∑Cd=1 θ

(s)ic θ

(s)jd π

(s)cd . However, instead

of computing NPMI and coherence, we ran link-predictionexperiments. Specifically, prior to inference, we held outall elements of the count matrix that involved the top fiftysenders or recipients. After inference, we used the savedsamples to compute µij , but only for the held-out elements,and then calculated the corresponding reconstruction error.

Results. We find that our method generally obtains lowerreconstruction and lower held-out error than either thenaıve approach or non-private Poisson factorization of thenon-privatized data. The results for C∈{5, 20} are shownin figure 5. We provide results for C=10 in the appendix.

6. Conclusion and future workWe presented a general and modular method for privacy-preserving Bayesian inference for Poisson factorization.Our method satisfies limited-precision local privacy, a gen-eralization of local differential privacy that we introducedto formulate appropriate privacy guarantees for sparse countdata. In two case studies, we demonstrated that our methodgenerally outperforms the commonly used naıve approach,wherein inference treats the privatized data as if it werenot privatized. Surprisingly, our method also outperformednon-private Poisson factorization. In the context of topicmodeling, it inferred more coherent topics; in the contextof overlapping community detection, it obtained lower held-out error in link-prediction experiments. These findingsare consistent with known connections between privacy-preserving mechanisms and regularization (Chaudhuri &Monteleoni, 2009). Because our method can attribute smallcounts to the added noise, it can therefore ignore them. Inturn, it may be less susceptible to inferring spurious struc-ture. Initial results suggest that yi ∼ Skel(λ(+)

i +µi, λ(−)

i )may be a robust alternative to non-private Poisson factoriza-tion. We therefore highlight this direction for future work.


AcknowledgmentsWe acknowledge the support of NSF awards #1812699 and#1652536. We also thank Dan Sheldon, David Mimno, andAntti Honkela. This research was partially conducted whileAlexandra Schofield was a summer intern at Microsoft.

ReferencesAcharya, A., Ghosh, J., and Zhou, M. Nonparametric

Bayesian factor analysis for dynamic count matrices.arXiv:1512.08996, 2015.

Andres, M. E., Bordenabe, N. E., Chatzikokolakis, K., andPalamidessi, C. Geo-indistinguishability: Differentialprivacy for location-based systems. In Proceedings ofthe 2013 ACM SIGSAC Conference on Computer &Communications Security, CCS ’13, pp. 901–914, NewYork, NY, USA, 2013. ACM. ISBN 978-1-4503-2477-9.

Ball, B., Karrer, B., and Newman, M. E. J. Efficient andprincipled method for detecting communities in networks.Physical Review E, 84(3):036103, 2011.

Bernstein, G. and Sheldon, D. R. Differentially privatebayesian inference for exponential families. In Advancesin Neural Information Processing Systems, pp. 2919–2929, 2018.

Bernstein, G., McKenna, R., Sun, T., Sheldon, D., Hay,M., and Miklau, G. Differentially private learning ofundirected graphical models using collective graphicalmodels. arXiv preprint arXiv:1706.04646, 2017.

Blei, D. M., Ng, A. Y., and Jordan, M. I. Latent Dirichletallocation. Journal of Machine Learning Research, 3:993–1022, 2003.

Buntine, W. and Jakulin, A. Applying discrete PCA indata analysis. In Proceedings of the 20th Conference onUncertainty in Artificial Intelligence, pp. 59–66, 2004.

Canny, J. GaP: a factor model for discrete data. In Pro-ceedings of the 27th Annual International ACM SIGIRConference on Research and Development in InformationRetrieval, pp. 122–129, 2004.

Cemgil, A. T. Bayesian inference for nonnegative matrixfactorisation models. Computational Intelligence andNeuroscience, 2009.

Charlin, L., Ranganath, R., McInerney, J., and Blei, D. M.Dynamic Poisson factorization. In Proceedings of the 9thACM Conference on Recommender Systems, pp. 155–162,2015.

Chaudhuri, K. and Monteleoni, C. Privacy-preserving lo-gistic regression. In Advances in Neural InformationProcessing Systems, pp. 289–296, 2009.

Chi, E. C. and Kolda, T. G. On tensors, sparsity, and non-negative factorizations. SIAM Journal on Matrix Analysisand Applications, 33(4):1272–1299, 2012.

Devroye, L. Simulating Bessel random variables. Statistics& Probability Letters, 57(3):249–257, 2002.

Dimitrakakis, C., Nelson, B., Mitrokotsa, A., and Rubin-stein, B. I. P. Robust and private Bayesian inference. InInternational Conference on Algorithmic Learning The-ory, pp. 291–305, 2014.

Dimitrakakis, C., Nelson, B., Zhang, Z., Mitrokotsa, A., andRubinstein, B. Differential privacy for bayesian inferencethrough posterior sampling. Journal of Machine LearningResearch, 18(11):1–39, 2017.

Dwork, C., McSherry, F., Nissim, K., and Smith, A. Cal-ibrating noise to sensitivity in private data analysis. InProceedings of the 3rd Theory of Cryptography Confer-ence, volume 3876, pp. 265–284, 2006.

Flood, M., Katz, J., Ong, S., and Smith, A. Cryptographyand the economics of supervisory information: Balancingtransparency and confidentiality. 2013.

Foulds, J., Geumlek, J., Welling, M., and Chaudhuri, K. Onthe theory and practice of privacy-preserving Bayesiandata analysis. 2016.

Geumlek, J. and Chaudhuri, K. Profile-based privacy for lo-cally private computations. CoRR, abs/1903.09084, 2019.URL http://arxiv.org/abs/1903.09084.

Ghosh, A., Roughgarden, T., and Sundararajan, M. Uni-versally utility-maximizing privacy mechanisms. SIAMJournal on Computing, 41(6):1673–1693, 2012.

Gopalan, P. K. and Blei, D. M. Efficient discovery of over-lapping communities in massive networks. Proceedingsof the National Academy of Sciences, 110(36):14534–14539, 2013.

Grimmer, J. and Stewart, B. M. Text as data: The promiseand pitfalls fo automatic content analysis methods forpolitical texts. Political Analysis, pp. 1–31, 2013.

Karwa, V., Slavkovic, A. B., and Krivitsky, P. Differentiallyprivate exponential random graphs. In International Con-ference on Privacy in Statistical Databases, pp. 143–155.Springer, 2014.

Klimt, B. and Yang, Y. The enron corpus: A new dataset foremail classification research. In European Conference onMachine Learning, pp. 217–226. Springer, 2004.

Lau, J. H., Newman, D., and Baldwin, T. Machine readingtea leaves: Automatically evaluating topic coherence and

http://arxiv.org/abs/1903.09084


topic model quality. In Proceedings of the 14th Con-ference of the European Chapter of the Association forComputational Linguistics, pp. 530–539, 2014.

Mimno, D., Wallach, H. M., Talley, E., Leenders, M., andMcCallum, A. Optimizing semantic coherence in topicmodels. In Proceedings of the Conference on EmpiricalMethods in Natural Language Processing, pp. 262–272.Association for Computational Linguistics, 2011.

Mohr, J. and Bogdanov, P. (eds.). Poetics: Topic Modelsand the Cultural Sciences, volume 41. 2013.

Narayanan, A. and Shmatikov, V. De-anonymizing socialnetworks. In Proceedings of the 2009 30th IEEE Sympo-sium on Security and Privacy, pp. 173–187, 2009.

Newman, D., Asuncion, A., Smyth, P., and Welling, M. Dis-tributed algorithms for topic models. Journal of MachineLearning Research, 10(Aug):1801–1828, 2009.

Paisley, J., Blei, D. M., and Jordan, M. I. Bayesian non-negative matrix factorization with stochastic variationalinference. In Airoldi, E. M., Blei, D. M., Erosheva, E. A.,and Fienberg, S. E. (eds.), Handbook of Mixed Member-ship Models and Their Applications, pp. 203–222. 2014.

Papadimitriou, A., Narayan, A., and Haeberlen, A. Dstress:Efficient differentially private computations on dis-tributed data. In Proceedings of the Twelfth EuropeanConference on Computer Systems, EuroSys ’17, pp. 560–574, New York, NY, USA, 2017. ACM. ISBN 978-1-4503-4938-3.

Park, M., Foulds, J., Chaudhuri, K., and Welling, M. Privatetopic modeling. arXiv:1609.04120, 2016.

Pritchard, J. K., Stephens, M., and Donnelly, P. Inferenceof population structure using multilocus genotype data.Genetics, 155(2):945–959, 2000.

Ramage, D., Rosen, E., Chuang, J., Manning, C. D., andMcFarland, D. A. Topic modeling for the social sciences.In Neural Information Processing Systems Workshop onApplications for Topic Models, 2009.

Ranganath, R., Tang, L., Charlin, L., and Blei, D. Deepexponential families. In Proceedings of the 18th Interna-tional Conference on Artificial Intelligence and Statistics,pp. 762–771, 2015.

Roberts, M. E., Stewart, B. M., Tingley, D., and Airoldi,E. M. The structural topic model and applied socialscience. In Neural Information Processing Systems Work-shop on Topic Models: Computation, Application, andEvaluation, 2013.

Schein, A., Paisley, J., Blei, D. M., and Wallach, H.Bayesian Poisson tensor factorization for inferring mul-tilateral relations from sparse dyadic event counts. InProceedings of the 21st ACM SIGKDD International Con-ference on Knowledge Discovery and Data Mining, pp.1045–1054, 2015.

Schein, A., Wallach, H., and Zhou, M. Poisson–gammadynamical systems. In Advances in Neural InformationProcessing Systems 29, pp. 5005–5013, 2016a.

Schein, A., Zhou, M., Blei, D. M., and Wallach, H. BayesianPoisson Tucker decomposition for learning the structureof international relations. In Proceedings of the 33rdInternational Conference on Machine Learning, 2016b.

Schmidt, M. N. and Morup, M. Nonparametric Bayesianmodeling of complex networks: an introduction. IEEESignal Processing Magazine, 30(3):110–128, 2013.

Skellam, J. G. The frequency distribution of the differencebetween two Poisson variates belonging to different pop-ulations. Journal of the Royal Statistical Society, SeriesA (General), 109:296, 1946.

Titsias, M. K. The infinite gamma–Poisson feature model.In Advances in Neural Information Processing Systems21, pp. 1513–1520, 2008.

Wang, Y.-X., Fienberg, S., and Smola, A. Privacy for free:posterior sampling and stochastic gradient Monte Carlo.In Proceedings of the 32nd International Conference onMachine Learning, pp. 2493–2502, 2015.

Warner, S. L. Randomized response: a survey technique foreliminating evasive answer bias. Journal of the AmericanStatistical Association, 60(309):63–69, 1965.

Welling, M. and Weber, M. Positive tensor factorization.Pattern Recognition Letters, 22(12):1255–1261, 2001.

Williams, O. and McSherry, F. Probabilistic inference anddifferential privacy. In Advances in Neural InformationProcessing Systems 23, pp. 2451–2459, 2010.

Yang, X., Fienberg, S. E., and Rinaldo, A. Differential pri-vacy for protecting multi-dimensional contingency tabledata: Extensions and applications. Journal of Privacyand Confidentiality, 4(1):5, 2012.

Yuan, L. and Kalbfleisch, J. D. On the Bessel distributionand related problems. Annals of the Institute of StatisticalMathematics, 52(3):438–447, 2000.

Zhang, Z., Rubinstein, B. I. P., and Dimitrakakis, C. On thedifferential privacy of Bayesian inference. In Proceedingsof the 30th AAAI Conference on Artificial Intelligence, pp.2365–2371, 2016.


Zhou, M. Infinite edge partition models for overlappingcommunity detection and link prediction. In Proceed-ings of the 18th Conference on Artificial Intelligence andStatistics, pp. 1135–1143, 2015.

Zhou, M. and Carin, L. Augment-and-conquer negativebinomial processes. In Advances in Neural InformationProcessing Systems 25, pp. 2546–2554, 2012.

Zhou, M., Cong, Y., and Chen, B. The Poisson gamma beliefnetwork. In Advances in Neural Information ProcessingSystems 28, pp. 3043–3051, 2015.

Appendix for Locally Private Bayesian Inference for Count Models

Aaron Schein 1 Zhiwei Steven Wu 2 Alexandra Schofield 3 Mingyuan Zhou 4 Hanna Wallach 5

1. Supplementary results1.1. Community detection C ∈ {5, 10, 20} results

123

ε/N = ln(1/α)

0.6

0.7

0.8

0.9

1.0

1.1

1.2

1.3

Rec

onst

ruct

ion

MA

E


Model Type

Naıve

Proposed

Non-private

123

ε/N = ln(1/α)


123

ε/N = ln(1/α)


123

ε/N = ln(1/α)

7.0

8.0

9.0

10.0

11.0

12.0

Hel

d-ou

tM

AE


Model Type

Naıve

Proposed

Non-private

123

ε/N = ln(1/α)


123

ε/N = ln(1/α)


Figure 1. Each subplot compares our method to the two reference methods for increasing levels of privacy. The top subplots depictreconstruction error (lower values are better); the bottom subplots depict held-out reconstruction error (lower values are better).

2. Proof of geometric mechanism as LPLP mechanismTheorem 1. Let randomized response methodR(·) be the geometric mechanism with parameter α. Then for any positiveinteger N , and any pair of observations y, y′ ∈ Y such that ‖y − y′‖1 ≤ N ,R(·) satisfies

P (R(y) ∈ S) ≤ eεP (R(y′) ∈ S) (1)

for all subsets S in the range ofR(·), where

ε = N ln( 1α

). (2)

Therefore, for any positive integer N , the geometric mechanism with parameter α is an (N, ε)-private randomized responsemethod with ε = N ln ( 1

α ). If ε′

N ′ =εN , then the geometric mechanism with parameter α is also (N ′, ε′)-private.

Proof. It suffices to show that for any integer-valued vector o ∈ Zd, the following inequality holds for any pair of


observations y, y′ ∈ Y ⊆ Zd such that ‖y − y′‖1 ≤ N :

exp(−ε) ≤ P (R(y) = o)

P (R(y′) = o)≤ exp(ε), (3)

where ε = N ln(1α

).

Let ν denote a d-dimensional noise vector with elements drawn independently from 2Geo(α). Then,

P (R(y) = o)

P (R(y′) = o)=

P (ν = o− y)P (ν = o− y′) (4)

=

∏di=1

1−α1+α α

|oi−yi|∏di=1

1−α1+α α

|oi−y′i|(5)

= α(∑d

i=1 |oi−yi|−|oi−y′i|). (6)

By the triangle inequality, we also know that for each i,

− |yi − y′i| ≤ |oi − yi| − |oi − y′i| ≤ |yi − y′i|. (7)

Therefore,

− ‖y − y′‖1 ≤d∑i=1

(|oi − yi| − |oi − y′i|) ≤ ‖y − y′‖1. (8)

It follows that

α−N ≤ P (R(y) = o)

P (R(y′) = o)≤ αN . (9)

If ε = N ln(1α

), then we recover the bound in equation 3.

3. Proof of two-sided geometric noise as exponentially-randomized Skellam noiseTheorem 2. A two-sided geometric random variable τ ∼ 2Geo(α) can be generated as follows:

τ ∼ Skel(λ(+), λ(−)), λ(∗) ∼ Exp( α1−α ), (10)

where Exp(·) and Skel(·) are the exponential and Skellam distributions. The latter is the marginal distribution of thedifference τ := g(+)−g(−) of two independent Poisson random variables g(∗) ∼ Pois(λ(∗)), where ∗ ∈ {+,−}.

Proof. A two-sided geometric random variable τ ∼ 2Geo(α) can be generated by taking the difference of two independentand identically distributed geometric random variables:1

g(+) ∼ Geo(α), g(−) ∼ Geo(α), τ := g(+) − g(−). (11)

The geometric distribution is a special case of the negative binomial distribution, with shape parameter equal to one (Johnsonet al., 2005). Furthermore, the negative binomial distribution can be represented as a mixture of Poisson distributions with agamma mixing distribution. We can therefore re-express equation 11 as follows:

λ(+) ∼ Gam(1, α1−α ), λ

(−) ∼ Gam(1, α1−α ), g

(+) ∼ Pois(λ(+)), g(−) ∼ Pois(λ(−)), τ := g(+) − g(−). (12)

Finally, a gamma distribution with shape parameter equal to one is an exponential distribution, while the signed difference oftwo independent Poisson random variables is marginally a Skellam random variable (Skellam, 1946). We therefore recoverthe generative process in equation 10. We visualize the Skellam and two-sided geometric distributions in figure 2.

1A video of Unnikrishna Pillai deriving this is available at https://www.youtube.com/watch?v=V1EyqL1cqTE.

https://www.youtube.com/watch?v=V1EyqL1cqTE


−80 −60 −40 −20 0 20 40 60 80

τ

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

P(τ|λ

1,λ

2)Skellam distribution

λ1=5, λ2=5

λ1=100, λ2=100

λ1=20, λ2=1

λ1=30, λ2=40

λ1=10, λ2=50

−80 −60 −40 −20 0 20 40 60 80

τ

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

P(τ|α

)

Two-sided geometric distribution

α=0.90

α=0.75

α=0.50

Figure 2. The two-sided geometric distribution (right) can be obtained by randomizing the parameters of the Skellam distribution (left).With fixed parameters, the Skellam distribution can be asymmetric and centered at a value other than zero; however, the two-sidedgeometric distribution is symmetric and centered at zero. It is also heavy tailed and the discrete analog of the Laplace distribution.

4. Proof of relationship between the Bessel and Skellam distributionsTheorem 3. Consider two Poisson random variables y1 ∼ Pois(λ(+)) and y2 ∼ Pois(λ(−)). Their minimumm := min{y1, y2} and their difference τ := y1 − y2 are deterministic functions of y1 and y2. However, if not conditionedon y1 and y2, the random variables m and τ can be marginally generated as follows:

τ ∼ Skel(λ(+), λ(−)), m ∼ Bes(|τ |, 2

√λ(+)λ(−)

). (13)

Proof. We begin by writing out the raw joint probability of y1 and y2:

P (y1, y2) = Pois(y1;λ(+))Pois(y2;λ(−)) (14)

=(λ(+))y1

y1!e−λ

(+) (λ(−))y2

y2!e−λ

(−)

(15)

=(√λ(+)λ(−))y1+y2

y1! y2!e−(λ

(+)+λ(−))

(λ(+)

λ(−)

)(y1−y2) / 2

. (16)

If y1 ≥ y2, then

P (y1, y2) =(√λ(+)λ(−))y1+y2

Iy1−y2(2√λ(+)λ(−)) y1! y2!

e−(λ(+)+λ(−))

(λ(+)

λ(−)

)(y1−y2) / 2

Iy1−y2(2√λ(+)λ(−)) (17)

= Bes(y2; y1 − y2, 2

√λ(+)λ(−)

)Skel(y1 − y2;λ(+), λ(−)); (18)

otherwise

P (y1, y2) =(√λ(+)λ(−))y1+y2

Iy2−y1(2√λ(+)λ(−)) y1! y2!

e−(λ(+)+λ(−))

(λ(−)

λ(+)

)(y2−y1) / 2

Iy2−y1(2√λ(+)λ(−)) (19)

= Bes(y1; y2 − y1, 2

√λ(+)λ(−)

)Skel(y2 − y1;λ(−), λ(+))

= Bes(y1;−(y1 − y2), 2

√λ(+)λ(−)

)Skel(y1 − y2;λ(+), λ(−)). (20)

Ifm := min{y1, y2}, τ := y1 − y2, (21)

then we can change variables using the deterministic relationships

y2 = m, y1 = m+ τ if τ ≥ 0 (22)y1 = m, y2 = m− τ otherwise. (23)


The Jacobean can be computed as ∣∣∣∣ ∂y1∂m

∂y1∂τ

∂y2∂m

∂y2∂τ

∣∣∣∣ = ∣∣∣∣ 1 11 0

∣∣∣∣τ≥0 ∣∣∣∣ 1 01 −1

∣∣∣∣τ<0

= 1, (24)

so

P (m, τ) = P (y1, y2)

∣∣∣∣ ∂y1∂m

∂y1∂τ

∂y2∂m

∂y2∂τ

∣∣∣∣= Bes

(m; |τ |, 2

√λ(+)λ(−)

)Skel(τ ;λ(+), λ(−)). (25)

ReferencesJohnson, N. L., Kemp, A. W., and Kotz, S. Univariate discrete distributions. 2005.

Skellam, J. G. The frequency distribution of the difference between two Poisson variates belonging to different populations.Journal of the Royal Statistical Society, Series A (General), 109:296, 1946.

Documents

Locally Private Bayesian Inference for Count Models · Poisson factorization of the non-privatized data, suggesting that non-private Poisson factorization may be overﬁtting. 2