Population Size Estimation

and coverage calculation

for MARPs and MARA

Dave Burrows, DirectorAIDS Projects Management Group

““Coverage”Coverage” Perhaps the most mis-named, misused and least understood

concept in HIV work

Coverage means whatever the person using it chooses to mean

Most common use: % of people ever reached (or reached in 1 year) with an intervention: this is an utterly useless statistic

If 100% of IDUs are reached once with education or a new needle & syringe, or if MSM or SW are reached once with education or a condom, it will have virtually no impact on a HIV epidemic

11stst problem is PSE problem is PSE PSE increasingly needed for national HIV plans & GF

projects: if do not know size of population, how can we estimate coverage after 5 or 6 years of programs + plan scale-up?

Whatever definition of coverage is used, it almost always begins with “% of X population (IDUs, MARA, etc)”

X population is the denominator for all further calculations related to coverage and its constituent parts: reach, regularity of reach, breadth of services, quality

To find X population, population size estimation (PSE) methods are used

Why is PSE so difficult?Why is PSE so difficult? Some populations difficult to count, especially

hidden, stigmatised

Usual epidemiological methods such as national household or schools surveys usually do not work

Definition problems: eg, IDU has “ever injected”, “injected in past month”, injected in past year”?

Even more difficult for MARA and MARY as most epidemiological statistics & estimates are not disaggregated by age (or sex)

PSEPSE Methods Methods Variety of methods available, but most include: Consensus/ Delphi Multiplier methods

Other potential methods RDS: Respondent Driven Sampling Social networks

Consensus/ DelphiConsensus/ Delphi Asks key informants to agree on number of people

in X population Can be done at: National level All levels from local to national

Local to national seems to generate most accurate numbers

Should be triangulated with other methods

Multiplier methodsRecommended by UNAIDS for population

size estimation, eg for reporting on UNGASS IDU indicator

Uses existing data source with survey data

Benchmark: Reliable, regularly collected data: IDUs accessing health services, drug treatment, overdose deaths

Multiplier: Survey of as broad a sample as possible (eg not just from treatment centres)

Multiplier formulaMultiplier formulaX (population) = multiplier x benchmark

Example: 1000 IDUs entered drug treatment in 2007 (benchmark)

10% of IDUs surveyed said they entered drug treatment in 2007 (multiplier)

X = 1000 x 10/100 (= 10)

X = 10,000 IDUs

TriangulationSingle multiplier exercises tend to be

inaccurate

UNAIDS recommends using 3 at least separate processes, and averaging results to find a mean estimate:

Eg: Different processes may give 10,000; 8000; 11,000. Mean = 9670

RDS/ Social networksRDS/ Social networks RDS uses snowball sampling in specific methods to

attempt to achieve highly representative sampling: was not developed as a PSE method!

Mexico AIDS Conference: meta-analysis of 200 RDS papers found no evidence that RDS is useful in PSE

Promoted by many agencies with little/no evidence of accuracy; costly, time-consuming

Social networks PSE: new method, currently promoted by UNAIDS PSE workshops. May have value but requires evaluation, and to date apears costly and time-consuming

APMG Tajikistan projectAPMG Tajikistan project In Tajikistan, APMG is finalising a 5-month process for

UNDP (GF PR) to: Estimate national populations of IDUs and SW Risk behaviour of IDUs & SW in 5 sites Capacity of implementation agencies to scale up service

delivery to IDUs & SW in these sites

In addition, APMG is trying to tie this process to ongoing PSE for IDUs and SW as numbers change (especially locally as IDUs & SW are chased from 1 area by police activity or attracted to an area by availability of drugs or SW clients)

Tajikistan PSE methodsTajikistan PSE methodsExpert estimation (Delphi) at rayon level,

combined at oblast and national levelsSurvey for risk assessment included

multiplier question re use of narcological services in 2008

Benchmark: narcological statistics in 2008Results presented to national consensus

meeting (September 21) to agree final numbers

Lessons learned from TajikistanLessons learned from TajikistanBiggest error was carrying out risk

assessment and PSE simultaneously: much larger sample sizes needed for risk assessment sampling meant expert estimation could not be carried out in all rayons nationally

PSE can be relatively cheap and quick if done as a stand-alone activity

Lessons learned from TajikistanLessons learned from Tajikistan Rayon-level estimation requires national/ oblast

level staff to assist local officials to come to consensus

Time should be included to allow rayon estimates to be considered at oblast level, then national meeting based on oblast estimates

If this process used, could set up 6-monthly monitoring by asking rayons to consider increases/ decreases over the previous 6 months. Requires oblast/ national compilation

Tajikistan lessons re MARATajikistan lessons re MARAPSE of MARA in Tajikistan could be

accomplished using the same methods (with lessons learned) BUT

Definition required Definition would need to be agreed with

officials from various departmentsDefinition to be explained at rayon level

ArmeniaArmenia PSE of MARA in Armenia carried out by APMG

and MoH staff working on GF RCC proposal (2008)

Had already estimated IDUs, MSM, SW, migrants, uniformed personnel

MoH wanted to include specific programs for MARA but this required a statement about projected coverage after 6 years

To calculate coverage figure, PSE was needed for MARA

Armenia MethodsArmenia Methods No time available for MARA PSE study Estimate figure was calculated as 5% of all

adolescents in Armenia on the basis of household and school surveys that showed at least 5% of adolescents engaging in risky sexual behaviour or illicit (not necessarily injecting) drug use

Population estimate was used in RCC proposal with a note that a full PSE would be carried out as part of the grant activities

RCC was approved and will begin in late 09

MacedoniaMacedonia PSE in Macedonia will be carried out by National

Public Health Institute (NPHI) for MoH (GF PR) NPHI has decided to combine PSE with risk

behaviour survey and to use RDS (against our advice)

APMG’s role will be to examine all documents (methods, instruments, sampling frames, data analysis & reports) to recommend corrections

From this process, we will be able to learn lessons about use of RDS for PSE (probably by end 09)

Some further thoughts on CoverageSome further thoughts on Coverage APMG accepts WHO Universal Access definition:

% of those who need an intervention who receive that intervention

APMG sees 3 aspects: Reach, including regularity of reach. What % of the total

population participate? Is this a sufficient proportion to prevent/ reverse/ treat the epidemic?

Breadth: Spectrum of Services. Are interventions able to prevent/ reverse/ treat the epidemic?

Quality: Are interventions sufficiently attractive and effective to meet their objectives?

CoverageCoverage Calculation Calculation APMG accepts WHO Universal Access definition:

% of those who need an intervention who receive that intervention

E.g., for needle-syringe programs, it appears that a percentage of IDUs in a specified area need to access NSP of adequate quality ON A REGULAR BASIS to prevent/ reverse a HIV epidemic among IDUs.

WHO, UNODC and UNAIDS state that the % of IDUs who have been reached by NSP regularly (at least monthly for past 12 months) should be considered as:

Low coverage: <20%Medium coverage : >20– <60%High coverage : >60%

CoverageCoverage questions questionsWHO, UNODC and UNAIDS Target

Setting Guide for IDUs include: Proportion of IDUs regularly reached by NSP Number of pharmacies/ 1000 IDUs NSP sites/ 1000 IDUs Number of syringes distributed per IDU per year % of IDUs who have been reached by NSP

regularly (at least monthly for past 12 months)

% of IDUs who have been reached by NSP in the past month

CoverageCoverage questions 2 questions 2 Similar questions on proportion of IDUs in

substitution treatment Similar questions on proportion of IDUs in

other drug dependence treatment Similar questions on proportion of IDUs

participating in VCT and know their results Ratio of HIV+ IDUs receiving ART to non-

IDU HIV+ receiving ART (relative to proportions of HIV+ population)

Questions on TB, hepatitis C, etc

QualityQuality Generally, view is that quality should be measured by

adherence to guidelines, e.g. target setting guide asks: Percentage of NSP sites adhering to WHO guidelines on

NSP Percentage of NSP sites adhering to UNAIDS best practice

recommendations for HIV prevention among IDUs Percentage of occasions when clients access an NSP and

receive IEC Percentage of occasions when clients access an NSP and

receive condoms In Russia, APMG is helping Russian Harm Reduction

Network to develop NSP quality measurement and improvement processes based on the WHO/ UNAIDS/ UNODC Guide to Starting and Managing NSPs

Manual plus instruments should be available in English & Russian early 2010

Coverage for other MARPsCoverage for other MARPs Similar processes now under way for MSM: APMG working with Amfar, UNDP & WHO on

coverage calculation, targets & breadth of services WHO working on similar processes re SW

MARA and MARY not yet really included in these global processes