UNITED NATIONS New York and Geneva, 2OO9

NOTE

The designations employed in this publication, which are in conformity with the practice of the publishing organizations, and the presentation of material therein do not imply the expression of any opinion whatsoever on the part of the publishing organizations concerning the legal status of any country, area or territory or of its authorities, or concerning the delimitation of its frontiers or boundaries.

Copyright United Nations, 2009

United Nations Economic Commission for Europe (UNECE) and

International Labour Office (ILO) International Monetary Fund (IMF)

Organisation for Economic Co-operation and Development (OECD) Statistical Office of the European Communities (Eurostat)

The World Bank Office for National Statistics (ONS), United Kingdom

All rights reserved Printed at United Nations, Geneva, Switzerland

ECE/CES/STAT/NONE/2009/2

iii

ForewordThisHandbook is targeted at developing countries focusing on practical solutions to the problems facingcompilersof the consumerprice index (CPI) in thedevelopingworld. Itprovidespracticalguidanceonallaspects of compiling a CPI, and offers advice on a range of operational issues in data collection, dataprocessing and publication. Although targeted at CPI compilers in developing countries itwill also be ofinteresttocompilersmoregenerally.TheHandbookfollowsandsupplementstheinternationalConsumerPriceIndexManual.TheoryandPractice,published in2004. It isaccompaniedbyaCDROMwith trainingaidsand references, including interactiveexamplesofthemethodsandcalculationsgivenintheHandbook,whereuserscaninputtheirowndata.Thepublicationof theHandbook is the resultofacollaborativeeffort involvingexperts fromanumberofcountriesandorganisations,andithasbenefitedfromtheinputfromCPIcompilersindevelopingcountries.ByreferringtotheCPIManualanddrawingonexperienceofexpertsfrombothdevelopedanddevelopingcountries the Handbook provides internationally agreed guidance and suggests good practices with theobjectivetohelpnationalstatisticalofficestofurtherimprovetheirCPIstatistics.

v

PrefaceIntroduction

Theproductionof thisHandbookhasbeen fundedby theUKDepartment for InternationalDevelopmentunder the direction of the UK Office forNational Statistics and developedwith support from the InterSecretariatWorkingGrouponPriceStatistics.Itisdesignedtogivepracticalguidanceonthecompilationofconsumerprice indices (CPIs), focusingonpractical solutions to issues facing the compilersofCPIs in thedevelopingworld.Itoffersadviceonawholerangeofoperationalissuesconfrontedbyindexcompilersfromthesamplingandcollectionofpricesthroughtoindexconstructionandfinalpublication.Althoughtargetedat compilers of CPIs in developing countries itwill also be of practical use to compilers of CPIs in othercountries.

ThisHandbooksupplementstheILOManualonConsumerPriceIndices.Thelatterprovidesacomprehensivereview of the economic and statistical theory which underlies CPIs, as well as an indepth guide tocompilation,andhasbeenwrittenforthebenefitofusersaswellasproducers. Incontrast,theHandbooktakestheunderlyingtheoryasgiven.CrossreferencestotheILOManualareprovidedforthosewhowishtoinvestigatethemoretheoreticalaspects.UsersofthisHandbookareencouragedtotakeadvantageofthisasitisimportanttounderstandingtheconceptualandtheoreticalcontextunderlyingthepracticaladvicebeinggiven.

TheHandbook isalsoaccompaniedbyasetofPowerPointslidesfor localtrainingpurposesandaCDROMwithinteractiveexamplesoftheillustrativecalculationsgivenintheHandbook,whereuserscaninputtheirowndata.

Background

In 2004 a comprehensive Consumer Price Index Manual: Theory and practice was published by theInternational Labour Office (ILO),with the support and cooperation of a number of other internationalorganisations.ItisanexpandedrevisionoftheearlierILOpublicationConsumerpriceindices:anILOmanual,publishedin1989.ThenewManualiswidelyregardedasthemaininternationalreferencesourceforCPIs.Itcoversvirtuallyeveryaspectofaconsumerprice index.YetsomeCPIpractitioners,especially indevelopingcountries, have found that its comprehensiveness has made it less suitable as a daytoday source ofguidance, particularly on detailed practical issues. This Handbook, which is a companion volume to theManual,focusesprimarilyonpracticalmatters,anddealslesswiththeorythandoestheManual.ItshouldbeseennotasareplacementfortheManualbutasasupplement.

In December 2003 the International Conference of Labour Statisticians, convened by the ILO, adopted aResolution concerning consumer price indices. This Resolution replaced the previous Resolution on CPIsadopted in 1987. The ILO Resolution and the CPI Manual are compatible documents, and indeed thepreparationofthedraftResolutionwasdoneinparallelwiththepreparationoftheManual.ThetextoftheILOResolutionisincludedasAppendix3totheCPIManualitself.

InplanningthepresentHandbook,itwasdecidedthatalthoughdesignedforusebyCPIpractitionersineverycountry,specialattentionshouldbepaidtotheneedsofdevelopingandtransitioncountries.Theauthorsofeachchapterweremadeawareofthispolicyfromthestart.TheILO,incollaborationwithUNECE,carriedoutausersurveyoftheCPIManualinordertoelicitusefulideasforinclusionintheHandbook,andtodiscovermore about the perceived limitations of theManual. The surveywas addressed to all national statisticalinstitutesandalso tootherusers suchasministriesof labourandworkersandemployersorganisations.Over100 responses fromasimilarnumberofcountrieswere received.More thanhalfof respondents feltthatmorespaceshouldbedevotedtopracticerelatedissuesandthemostusefulchapterintheManualwasthoughttobeChapter9onthepracticalaspectsofCPIcalculation.

PRACTICALGUIDETOPRODUCINGCONSUMERPRICEINDICES

vi

Giventhespecialfocusondevelopingcountries,itwasdecidedthatapanelofreviewersfromsuchcountriesshouldbe involved in thepreparationof theHandbook.More than tenCPIexperts froma representativegroupofcountrieswereaccordinglyasked tocommentonspecificchapters.Theirviewswereregardedasextremely valuable and the content of the Handbook has benefited as a result. For example, there arefrequentreferencestotheproblemsmetincollectingpricesinmarkets,whichtothereaderinadevelopedcountrymayseemdisproportionate,buttoCPIpractitionersindevelopingcountriesarecrucial.

ToassistreadersoftheHandbook,thechapterheadingshavebeenmadeascompatibleaspossiblewiththatoftheManual.Intermsofgeneralorderingthishasbeenachieved,thoughexactnumericalcompatibilityislimitedtojustafewofthe18chapters.

Withitsemphasisonpracticalmatters,theHandbookhasbeenwrittenwithaviewtousebyCPIstatisticiansworking innational statisticaloffices.Thevarious topicspecificchapters shouldbeuseful forpractitionersfacedwithspecificproblemsincertainareasofCPIcompilation(e.g.howtodealwithseasonalproductsorhowtoaggregatepricesatthe lowest levelwheretherearenoweightsavailable). It isalsohopedthattheHandbookcanbeusedfortrainingpurposes.The languageused,aswellasthestyle,hasbeendeliberatelyaimedatalevelwhichshouldbeeasilyunderstoodnotonlybyCPIexpertsbutalsobynewprofessionalstaffjoiningaCPIdepartmentinanationalstatisticalinstitute.AswiththeManualitself,anelectronicversionoftheHandbook is available on the internet at http://www.ilo.org/cpimanuals and is regarded as a livingdocument thatwillbe amended andupdated asnecessaryover the coming years.Asmentioned in theIntroduction, there is an accompanying CDROMwhich includes not onlyworked exampleswhich canbeadaptedusingnationaldatasets,but also the textof theCPIManual, and someotherusefulbackgroundpapers.

Aguidetoreaders

TheHandbookbroadly follows thesamesequenceofchaptersas in theManualso that itcovers in logicalsequencethesuccessivestepsinindexcompilation.Butitisnotdesignednecessarilytobereadfromcovertocover. It isexpectedthatcompilerswillbeabletodip into itasrequired.Eachchapter isselfcontained,giving a brief summary of the corresponding chapter in themain ILOManual before providing practicalguidance. Illustrativeexamplesofcalculationsandmodeldocumentsarealsopresented toassist theuser.TheHandbookusesthesameterminologyastheManual.FurtherdetailsonthecontentsoftheHandbookaregiveninChapter1.

The Handbook is not prescriptive where it cannot or does not need to be, and in these cases makesrecommendationsorgivesexamplesofwhatmightbeconsideredbestpractice.Thereadershouldbeartwothings inmind.Firstly, it isnotalwayspossible togivedefinitivepracticalguidanceas thiswouldwronglyassume that thesolutions toconceptualproblemsare, themselves,alwaysclearcutand that therearenochoices tobemadeaboutpreciselyhowapracticalsolution is implemented. Secondly,what isapplicablelocally andwhat canbe achievedon the groundwilldependon the resources available to the individualNationalStatisticalInstitute.

Acknowledgements

TheproductionofthisHandbookwasacollaborativeeffortinvolvingmanypeople.AuthorsoftheindividualchaptersandtheircoauthorsweredrawnfromCPIexpertsaroundtheworldandcontributionsweresoughtfromexpertswithextensiveexperienceinworkingonCPIissuesindevelopingcountries.TheHandbookalsodrew on documentation from the UK Office for National Statistics and from work undertaken by theStatistical Office of the European Communities (Eurostat) in connection with the development of theEuropeanUnionsHarmonisedIndexofConsumerPrices.ThelatteractedasafocalpointforimprovingthecompilationofCPIsintheEuropeanUnionandsignificantlyincreasedlevelsofunderstandingandexpertiseinCPImethodology.

DavidFenwickwastheEditorinChiefofthehandbookandaccompanyingCDROM.JohnAstinthecoeditoroftheHandbookandTerryOffnerthecoeditoroftheCDROM.Theauthorsand,inbrackets,coauthorsoftheindividualchaptersareasfollows:

PREFACE

vii

1.IntroductionDavidFenwickandJohnAstin2.TheScopeoftheCPIJohnAstin(PaulArmknecht)3.ProductClassificationMarcPrudHomme(JohnAstin)4.ExpenditureWeightsintheCPIMarcPrudHomme(PaulArmknecht)5.SamplingProceduresDavidFenwickandMatthewPowell(TerryOffner)6.PriceCollectionTerryOffner(JohnAstin)7.SubstitutionandQualityChangePaulArmknecht(WalterLane)8.NewProductsPaulArmknecht(MatthewPowell)9.1.SpecialCasesHousingDavidFenwick(YoelFinkel,JohnAstin)9.2.SpecialCasesOwnAccountProductionKeithBlackburn(JohnAstin)9.3.SpecialCasesServicesWalterLane(MarcPrudHomme)9.4.SpecialCasesTariffsMarcPrudHommeandDavidFenwick(WalterLane)9.5.SpecialCasesSeasonalProductsYoelFinkelandDavidFenwick(MarcPrudHomme)9.6.SpecialCasesSecondHandGoodsDavidFenwick(PaulArmknecht)10.IndexCalculationJohnAstinandMarcPrudHomme(DavidFenwick)11.SpecialIndicesJohnAstin(MarcPrudHomme)12.DataValidationTerryOffner(MatthewPowell)13.SourcesofErrorsandBiasinaCPIYoelFinkel(PaulArmknecht)14.Publication,Presentation,AnalysisandInterpretationDavidFenwick(JohnAstin)15.OrganisationandManagementDavidFenwickandYoelFinkel16.UserConsultationDavidFenwick(YoelFinkel)17.CPI/ICPIntegrationandHarmonisationJohnAstin(KeithBlackburn)18.QualityReportingandimprovingtheCPI:Frameworks,checklistsandworkprogrammes.DavidFenwick(GeorgeBeelen)19.GlossaryJohnAstin

ThequalityoftheHandbookwasalso increasedbythevaluablecontributionsofcolleagues, includinguserinput fromCPI compilers inanumberofnational statistical institutesparticularly indeveloping countries.Special thanks are due to staff at theAfricanDevelopment Bank and theAsianDevelopment Bank,whoprovideduseful comments, to theUNECE, ILO, the IMF,COMESA, andCEPALwho facilitated consultationwithCPIcompilersonearlierdrafts,andtothefollowing individualswhoprovidedsomehelpfulcommentswhichwere incorporated inthe final text: NelsonTaruvinga;VincentMusokeNsubuga;PatrickKelly;SaadBashir;PhillipMiti;SanjevBhonoo;NicholasNsowahNsowah;BorbalaMinary;MariettaMoradaandZakayoMsokwa.

Iwouldliketothankmycoeditorofthehandbook,JohnAstin,whoalsodraftedorcoauthoredanumberofchapters.IamalsogratefultoErwinDiewert,whotookonthetaskofreviewingthemoretechnicalaspectsofthehandbook,includingthescrupulouscheckingofformulaeforerrorsandinconsistencyinannotation.

TheideaofaCDROMevolvedduringtheearlystagesofdraftingofthehandbookandbecameaprojectinitsownright.IamindebtedtoTerryOffnerforapplyinghistechnicalskillsinundertakingmuchofthedetailedworkandforactingasmycoeditorontheCDROM,andtoMarcPrudHommefordraftingthePowerPointpresentations,whichareanintegralandimportantpartofthematerialsincludedintheCDROM.

Theproductionofaninternationalhandbook,whichreliesoncontributionsfromaroundtheworld,requiresagreatdealofcoordination.BenWhitestone (UKOfficeforNationalStatistics)tookontheroleofprojectmanagerandhiseffortswereasignificantfactorinthesuccessofthisproject.

Finally, thanks go to theUK Department for International Developmentwho funded the drafting of thisHandbook, theUKOffice forNational Statisticswhoprovidedagreatdealofday today supportand theUnitedNationsEconomicCommissionforEurope(UNECE)forundertakingtheprintinganddistribution.

DavidFenwickEditorinChief

ix

ContentsForeword ......................................................................................................................... iii Preface ............................................................................................................................. v Introduction.......................................................................................................................... v Background .......................................................................................................................... v Aguidetoreaders............................................................................................................... vi Acknowledgements............................................................................................................. vi

1Introduction................................................................................................................... 1 SystemsofPriceIndicesandsupportingframeworks......................................................... 1 LowerlevelFrameworks...................................................................................................... 2 Choiceofindexnumberformula ......................................................................................... 3 IndexFormulaatLower(ElementaryAggregate)Level ...................................................... 3 Acquisition,use,orpaymentapproach............................................................................... 3 ContentsoftheHandbook................................................................................................... 4 TerminologyintheHandbook ............................................................................................. 6

2TheScopeoftheCPI ...................................................................................................... 8 Introduction ......................................................................................................................... 8 ThedifferentusesofaCPI ................................................................................................... 8 Geographicalcoverage ........................................................................................................ 9 ReferencepopulationfortheCPI ...................................................................................... 10 Democraticversusplutocraticweights.............................................................................. 11 Productexclusions ............................................................................................................. 11 Imputedtransactionsandimputedprices......................................................................... 12

3ProductClassification .................................................................................................. 13 Introduction ....................................................................................................................... 13 Classificationsystems:thegeneralcase............................................................................ 13 TheCPIclassificationsystem ............................................................................................. 14 SpecificrequirementsofaCPIclassification ..................................................................... 14 TheCOICOPclassification .................................................................................................. 15 BenefitsofadoptingtheCOICOPclassification ................................................................. 16 NationalversionsofCOICOPatthesubclasslevel ........................................................... 16 IncorporatingnewproductsintheCPIclassification ........................................................ 17 ChangingfromanationalclassificationtoCOICOP ........................................................... 17

4ExpenditureWeightsintheCPI.................................................................................... 18 Introduction ....................................................................................................................... 18 Background ........................................................................................................................ 18 Conceptualbasisoftheweights ........................................................................................ 19 Weightingstructure ........................................................................................................... 23 Datasources....................................................................................................................... 25 NationalAccounts .............................................................................................................. 28 Weightsreferenceperiod .................................................................................................. 29

5SamplingProcedures ................................................................................................... 38 Introduction ....................................................................................................................... 38 Background ........................................................................................................................ 38

PRACTICALGUIDETOPRODUCINGCONSUMERPRICEINDICES

x

Overview ............................................................................................................................ 39 Illustrativeexampleofselectingasampleforpricecollection ......................................... 41

6PriceCollection ............................................................................................................ 48 Introduction ....................................................................................................................... 48 Background ........................................................................................................................ 48 Theprinciplesofpricecollection ....................................................................................... 49 Itemspecifications ............................................................................................................. 49 Thepricestobecollected.................................................................................................. 51 Problemssometimesencountered.................................................................................... 56 Frequencyandtimingofpricecollection .......................................................................... 56 Practicalcollectionprocedures:planningandorganisation ............................................. 58 Practicalcollectionprocedures:collectingpricesinthefield ........................................... 59 Trainingandworkinstructions .......................................................................................... 62

7SubstitutionandQualityChange.................................................................................. 71 Introduction ....................................................................................................................... 71 Background ........................................................................................................................ 71 Substitutionprocedures .................................................................................................... 72 Temporarily(nonseasonal)missingproducts................................................................... 73 Permanentlymissingproducts .......................................................................................... 73 Qualityadjustingthepricefordifferencesinquality ........................................................ 75 Directcomparison.............................................................................................................. 75 Directorexplicitqualityadjustmentmethods .................................................................. 75 Implicitqualityadjustment................................................................................................ 77 Matchedmodels ................................................................................................................ 81 Summary ............................................................................................................................ 81

8NewProducts .............................................................................................................. 88 Introduction ....................................................................................................................... 88 DefinitionofNewProducts................................................................................................ 88 Planningfortheintroductionofnewproducts ................................................................. 89 Timingoftheintroductionofnewproducts ..................................................................... 90 Methodsofincorporatingnewproductsintotheindex ................................................... 91 Geographicaldimension .................................................................................................... 95 ServicesPreviouslyProvidedFree ..................................................................................... 95

9.1SpecialCasesHousing ............................................................................................101 Introduction ..................................................................................................................... 101 Owneroccupierhousingcosts ........................................................................................ 101 Rentedaccommodation................................................................................................... 105

9.2SpecialCasesOwnAccountProduction..................................................................108 Introduction ..................................................................................................................... 108 Background ...................................................................................................................... 108 Theownaccountproductiondilemma............................................................................ 109 Goodsproducedforownconsumption........................................................................... 110 Constructionofageneralconsumptionindex ............................................................. 111 Constructionofamonetaryindex................................................................................ 112 Constructionofacompensationindex ........................................................................ 112 Ownaccounthouseholdservices(excludingowneroccupierhousingservices) ........... 112

9.3SpecialCasesServices ............................................................................................114

CONTENTS

xi

Introduction ..................................................................................................................... 114 Background ...................................................................................................................... 114 Health,educationandsocialprotectionservices............................................................ 115 Healthservices(doctorsanddentists) ............................................................................ 117 Education ......................................................................................................................... 120 Transport.......................................................................................................................... 122

9.4SpecialCasesTariffs ...............................................................................................123 Introduction ..................................................................................................................... 123 Background ...................................................................................................................... 123 Themainmeasurementissues ........................................................................................ 124 Treatmentoftariffsandtheinformationrequirement................................................... 124 Pricemeasurementmethods .......................................................................................... 125 Illustrativeexamples ........................................................................................................ 127 Qualityadjustment .......................................................................................................... 128 Newserviceprovidersandnewproducts ....................................................................... 131 Classifications................................................................................................................... 131

9.5SpecialCasesSeasonalProducts ............................................................................132 Introduction ..................................................................................................................... 132 Background ...................................................................................................................... 132 Fixedweightsapproach ................................................................................................... 133 Variableweightsapproach .............................................................................................. 134 Fixedorvariableweights? ............................................................................................... 135 Estimationofweights ...................................................................................................... 135 Monthlyapproach:maximumoverlapnotanalternative ........................................... 136 RollingYearIndexanalternativemethodofpresentation .......................................... 136 Somecommonimplementationproblems...................................................................... 137 Somecommonseasonalproductsandtheirtreatment.................................................. 138 Intractableproblemsandseriouschallenges? ................................................................ 139

9.6SpecialCasesSecondHandGoods .........................................................................140 Introduction ..................................................................................................................... 140 Qualityadjustment .......................................................................................................... 143 Alternativeapproaches.................................................................................................... 143

10IndexCalculation ......................................................................................................147 Introduction ..................................................................................................................... 147 Background ...................................................................................................................... 147 Thecompilationofindicesforelementaryaggregates ................................................... 148 Calculationformulaeforelementaryaggregateindices ................................................. 150 Thecompilationofupperlevelaggregateindices .......................................................... 156 LinkingandChaining ........................................................................................................ 161 Splicingandchainlinking................................................................................................. 163 Chainingandsplicingtochangetheweights................................................................... 163

11SpecialIndices ..........................................................................................................165 Introduction ..................................................................................................................... 165 Background ...................................................................................................................... 165 Seasonallyadjustedconsumerpriceindices................................................................... 165 Seasonaladjustmentmethods ........................................................................................ 166 Coreinflationandsmoothedindices............................................................................... 167

PRACTICALGUIDETOPRODUCINGCONSUMERPRICEINDICES

xii

Otherspecialindices........................................................................................................ 169 Illustrativeexamples ........................................................................................................ 170

12DataValidation .........................................................................................................176 Introduction ..................................................................................................................... 176 Background ...................................................................................................................... 176 Datavalidationandeditingproceduresinthefield ........................................................ 177 DatavalidationatRegionalorNationalHeadquarters.................................................... 178 Dataediting...................................................................................................................... 178 Nonstatisticalchecking ................................................................................................... 178 Statisticalchecking........................................................................................................... 179 Dealingwithoutliers ........................................................................................................ 181 Missingprices................................................................................................................... 182 Credibilitychecking.......................................................................................................... 182 Outputeditingofindices ................................................................................................. 183 Analyticaltools................................................................................................................. 183 Prioritisation..................................................................................................................... 184

13SourcesofErrorsandBiasinaCPI ............................................................................188 Introduction ..................................................................................................................... 188 Background ...................................................................................................................... 188 SummaryofGeneralMeasurementProblems................................................................ 188 BiasinaCPI ...................................................................................................................... 190 SamplingandNonsamplingErrors ................................................................................. 193

14Publication,Presentation,AnalysisandInterpretation .............................................196 Introduction ..................................................................................................................... 196 Timingofpublication ....................................................................................................... 196 Prereleaseaccess............................................................................................................ 197 Politicalandpolicystatementsbyministersandtheirofficials ...................................... 198 Proceduresonpublicationday ........................................................................................ 199 Formatandcontentofstatisticalpressnotice ................................................................ 199 Standardcalculationsandspecialpresentationsandanalysis........................................ 201 Internalpurchasingpowerofnationalcurrency ............................................................. 204 Rounding .......................................................................................................................... 205

15OrganisationandManagement ................................................................................211 Introduction ..................................................................................................................... 211 OverviewofQualityManagementSystem...................................................................... 211 Documentation ................................................................................................................ 212 InternalAudits.................................................................................................................. 214 Reviewsystems................................................................................................................ 216 Businesscontinuity .......................................................................................................... 219

16UserConsultation .....................................................................................................226 Introduction ..................................................................................................................... 226 Thegeneralprinciplesofconsultation ............................................................................ 226 SpecialarrangementsforconsultationontheCPI .......................................................... 227 Whotoconsultandonwhat ........................................................................................... 227 Practicaloptionsforconsultationadvisorycommittees................................................ 228 Advisorycommitteesoperationalarrangementsandtermsofreference ................... 229 AdvisorycommitteesroleoftheNSIandtheNationalStatistician .............................. 229

CONTENTS

xiii

Stakeholderengagementandmanagement ................................................................... 229 17CPI/ICPIntegrationandHarmonisation ....................................................................234 Introduction ..................................................................................................................... 234 Spatialversustemporalpriceindices .............................................................................. 234 BasicdatarelationshipsbetweentheICPandCPIs......................................................... 235 IntegrationofCPIandICPdatasets ................................................................................. 236 IntegrationbenefitsfromICPparticipation..................................................................... 237

18QualityReportingandImprovingtheCPI ..................................................................241 Introduction ..................................................................................................................... 241 Amodelframework:qualityreports ............................................................................... 241 Amodelframework:checklists........................................................................................ 242 Planning:workprogrammes............................................................................................ 242 ProjectManagement ....................................................................................................... 245

GlossaryofMainTerms .................................................................................................254

1

CHAPTER1

Introduction 1.1 Consumer price indices measure changesovertimeinthegenerallevelofpricesofgoodsandservicesthathouseholdsacquire,(useorpayfor)forthepurposeofconsumption.Inmanycountriestheywereoriginally introduced toprovideameasureofthechanges in the livingcosts facedbyworkers,sothatwageincreasescouldberelatedtothechanginglevelsofprices.However,over theyears,CPIshavewidenedtheirscope,andnowadaysarewidelyusedasamacroeconomic indicatorof inflation,asa toolby governments and central banks for inflationtargeting and formonitoring price stability, and asdeflators in the national accounts. With theglobalisation of trade and production and theliberalisationof themarkets,nationalgovernments,central banks and international organisations placegreat importance on the quality and accuracy ofnational CPIs, and in their internationalcomparability.

1.2 Differentconceptualframeworkscanbeusedtoaddress fundamental issues relating tonatureofthe index. For example, whether the CPI shouldmeasurethechangeincostofafixedweightbasketofgoodsandservicesorwhether itshouldmeasurethe change in the cost of living, i.e. the cost ofmaintaining a given standard of living, taking intoaccountthefactthatwhenpriceschangeconsumerschange their expenditure patterns. The use andconceptual basis of the index will determine themethodofconstruction,includingtheformulaeusedfortheaggregationofprices.

1.3 Themethod of construction also allows (orshouldallow)CPIstobeadaptedforawiderangeofspecificuses. Forexample, they canbe adapted tocalculate specific inflation rates for social groupssuchaspensioneror lowincomehouseholds.Theirproduct coverage can be adapted so as to showwhattherateofinflationisinparticularsectorssuchas energy or food, or excluding particular sectorssuchasalcoholandtobacco.Theycanshed lightonthe effect of tax changes or governmentregulatedprice changeson the rateof inflation.They canbecompiled on a regional basis, showing differentinflationrateswithindifferentpartsofacountryorbetweenurbanandruralareas.

1.4 In short, consumer price indices are nowconsideredasoneof themost importanteconomicandsocialindicatorsproducedbyNationalStatisticalInstitutes (NSIs) throughout theworld.Against thisbackground, the challenge of national statisticsinstitutes is fourfold: to identify user needs; toconceptualise user needs in terms of economictheory; to translate the underlying concept intostatistical measurement terms following thefundamentalprinciplesofpriceindexmeasurement;to construct the indices so defined and evaluatethemagainstpurpose.

SystemsofPriceIndicesandsupportingframeworks

Higherlevelframeworks

1.5 The System of National Accounts 2008provides the basic guidelines for buildingappropriate frameworks for CPIs and other priceindices in the familyofprice indices, covering allmajoreconomicactivity including,mostparticularly,production and consumption and intermediateoutputs. In theory theSystemofNationalAccountsalso provides a methodology for developing andextending the core systemofprice indices tomeetspecialised needswhilstmaintaining consistency ofapproach, both in concepts and practicalmeasurement, and coherence in terms of thedefinition, classification andmeasurement of flowsand stocks of goods and services. The use of suchframeworks also facilitates the examination of therelationships between different price inflationmeasures and can help analysts in theirunderstanding and interpretation of inflationarysignals.With theirability to identifystatisticalgaps,these frameworks can also facilitate futuredevelopments in the field of price statistics. Onesuch framework is the Stage of ProcessingFramework(seeAppendix1.1).

StageofProcessingFramework

1.6 A Stage of Processing framework has auseful role toplay in informingpolicy formation. Itcan also provide the basis for systematicallyanalysing the buildup of inflationary pressures in

PRACTICALGUIDETOPRODUCINGCONSUMERPRICEINDICES

2

theeconomyandfortrackingrelationshipsbetweenprice developments in particular sectors. From acompilerspointofviewitalsoprovidesausefultoolfor identifying gaps in available price index seriesandafocalpointforinvestigatingcoherenceinindexconstruction.

1.7 The first step in constructing such aframework is todivide theentireeconomicprocessintodistinctstages:

Restoftheworld

Production: split into broad categories such asagricultureandmanufacturing(andfurthersubdivided into primary, intermediate and finalproduction).

Final demand: separated into private finalconsumption,governmentconsumption,capitalpurchasesandexports.

1.8 The main series of price indices are thengrouped according to coverage. For instance, thismightbeasfollows:

Consumers expenditure (Consumer PricesIndex).

Other elements of Final Demand (an Index ofGovernment Prices [IGP] and an Index ofInvestmentPrices[IIP].

Inputsintoandoutputsfromthemanufacturingsector(theProducerPricesIndex[PPI]).

Tradepriceindices

1.9 In principle, for all significant transfers ofgoodsandservicesfromonepartoftheeconomytoanother, there should be a representative priceindex (based on directlymeasured prices) thatreflects the changing level of prices for all suchgoods(orservices).

1.10 TheStageofProcessingFramework isshowninAppendix1.1.Thesourceofthegoods/services isshown down the left side of the table; thedestination isshownalongthetopofthetable.Forinstance, the prices charged by themanufacturingsector for consumer goods sold to theretail/wholesale sector are reflected in the PPIs(ProducerPrice Indices). Theprices chargedby theretail sector for consumer goods sold to privateconsumersarereflectedintheCPI.Acrossthetopofthe table are the buyers, classified into two broadgroups: intermediate demand and final demand.Most sectors of production are involved in bothbuying inputsandsellingoutput sotheyappear inthetablebothasbuyersandassellers.Inpractice,insome cells transactions from seller tobuyerwillbeinsignificant and these are denoted by a dash. For

instance, there are unlikely to be any importedgoods feeding into the construction sector asintermediate demand. In the example in Appendix1.1, which for illustrative purposes has beenpopulated on the basis of a typical situation, theshadedcellsrepresentsignificanttransactionsintheeconomy for which there are no relevant priceindices.Wherepublished indicesdoexist,thename(oracronym)ofthe index isgiven.For instance,thesale of finished goods by retailers to private finalconsumptionismeasuredbytheCPI.

1.11 Thepractical advantagesofusing a StageofProcessingFrameworkcanclearlybeillustrated:

The associated table of inflation rates can beused for analytical purposes and help informeconomicpolicy.

Aggregateprice indices (and inflation rates) foreach row and for each column may becomputed to produce inflation rates fordifferentsectors.

Possible enhancements to currently publishedindicescanbeidentified.

Majorgapscanbeidentified.

Issues relating to statistical integration andcoherence and can be also identified. Forexample, issues relating to coherence inconceptsandpracticalmeasurementarisewhencombiningrowsandcolumns.

LowerlevelFrameworks

1.12 Higherlevel frameworks can besupplemented by lowerlevel frameworks such asthe Stage of Production Framework. The latter,whichcanbeused for indepth investigationof theProducerPriceIndex,isnotexpandeduponhere.

1.13 The lowerlevel theoretical framework forCPIsisessentiallybasedoneconomictheoryrelatingtoconsumerbehaviour.InthiscontexttheSchultzePanel on Conceptual, Measurement, and OtherStatisticalIssuesinDevelopingCostofLivingIndexesmakes a distinction between the traditional CPIbasedona fixedweightbasket,asbeinga costofgoodsindex(COGI),andacostoflivingindex,whichmeasures the change in expenditures a householdwould have tomake in order tomaintain a givenstandardof livingorutility.Many references in theplentiful literatureon consumerprice indicesarguethatanadvantageofaCOLIoveraCOGI isthattheformer is supported by economic theory (i.e. thebehaviour of the individual in the market place),implyingthatthe latter isnot.ButeconomictheorycanbeusedtosupportbothaCOGIandaCOLI,one

INTRODUCTION

3

atamacrolevelandoneatamicrolevel,andbothcomplementoneanother.Thus:

A COGImeasures the inflationary pressures inthe economy from price developments in theretailsector. Itrepresentsoneofmanysectors,albeit an important one, in the Stage ofProcessingFramework.

ACOLImeasurestheexpenditurerequiredbyahouseholdtomaintaintheirstandardoflivingorutility.Puttingasidethedefinitionofutilityandwhether the COLI is unconstrained orconstrained, it isan indexconstructedfromtheviewpointoftheindividualconsumerandhasitsfoundation inmicroeconomics and the theoryof individual consumer behaviour. Unlike aCOGI, it takes into account the substitutionsconsumers make when faced by relativechanges in prices either between differentgoodsandservicesorbetweendifferentoutletsandsuppliers.

1.14 The different uses of a CPI setwithin theseframeworks determine a number ofmeasurementissuesincludingthechoiceofindexnumberformula.

Choiceofindexnumberformula

1.15 Most countries state that they use aLaspeyresindexoraLaspeyrestypeindexfortheirnationalCPIwhich, inpractice, issomewherealongthe continuum between a costofgoods index(COGI)andacostoflivingindex(COLI).

1.16 Expertsgenerallyagreethatthe idealtypeofindex foraCPIwouldbeasuperlative indexsuchas the Fisher index. Superlative indicesmakeequaluse of the prices and quantities in both of theperiods being compared (such as the base periodand the latest month). In reality, quantities forrecent periods are almost never known, so that inpracticenearly allNSIsproduceCPIswhich relyonquantities relating to a base period some timeearlier.

1.17 Some countries aim to produce a cost ofliving index (COLI). But such an index is in fact atypeofsuperlative indexandsuffersfromthesamepracticaldefectasmentionedabove.

1.18 It is important, nevertheless, for nationalstatistical offices to be able to state publiclywhattypeof index isbeingcalculated intheirCPI.AtrueLaspeyres indexusesquantitydatawhich relate toexactly the same period as the price referenceperiod.Thisisrarelythecase.Moststatisticalofficeshaveapricereferenceperiodwhichislaterthantheperiod towhich thequantitydata (i.e. theweights)

relateandwillspan,say,ayearratherthanapointintime. This is because the main source of weightsdata is a Household Budget Survey (HBS) whichtypicallyproducesusableresultsayearormoreafterthe end of a survey periodwhich isnot a point intime. In thesecircumstances,either theHBSperiodweights are usedwithout adjustment, or they arepriceupdated to the price reference period. Eitherof these methods results in a Lowe index, but isoften referred to as a Laspeyrestype index. TheCPIManual(Chapter1)goesintoconsiderabledetailontheseandotherindexnumbertypes.

1.19 The Handbook in general is based on theassumptionthatthe indexbeingcompiledisaLowe(orLaspeyrestype)index.

IndexFormulaatLower(ElementaryAggregate)Level

1.20 ThefirststageinthecalculationofCPIsisthecalculation of elementary price indices, which arethenaggregatedtoobtainhigherlevelpriceindices.Expenditureweightsarenotusuallyavailablebelowthe elementary aggregate level. The three mostwidely known elementary index formulae are theCarli, the Dutot and the Jevons and each isassociatedwithanumberofassumptionswhichwillimpact on measured inflation. The Carli (a simplearithmeticaverageofpricerelatives)andDutot(theratio of simple arithmetic averages of prices)formulae have a number of problems associatedwith their use particularly the Carli, which ispositivelydiscouragedas it isparticularlyassociatedwith some bad characteristics. Because of this theJevons formula (the ratio of simple geometricaverages) is increasingly used. It should be notedthatanarithmeticaverage isalwaysgreaterthanorequaltoageometricaverageandthatthedifferencewillbegreater thegreater thevariance in thepricerelatives. The choice of formula becomes moreimportant the greater the diversity of pricemovementswhichisoneargumentforensuringthatelementary aggregates are as homogeneous aspossible.

Acquisition,use,orpaymentapproach

1.21 A CPI is based on themeasurement of thechange inpricesof thegoodsandservices includedin the basket. The vastmajority of goods (but notnecessarily of total values) are priced in the retailoutlets selling them. It should be noted that innormal circumstances the prices recorded are thelabelledprices,whichareassumed tobe theprices

PRACTICALGUIDETOPRODUCINGCONSUMERPRICEINDICES

4

actually paid by consumers. It is also generallyassumedthatpaymentforthegoodsismadeatthetime of purchase indeed the consumer wouldregard the two events as identical. However,paymentcanbeincashoroncredit,includingcreditcards forwhich the due date of paymentmay beseveralweeksaftertheactualpurchase.

1.22 The time factor is important in other waystoo. A shopper may decide to buy a larger thannormal quantity of a particular good if there is aspecial price reduction. The product may then bestored at home and consumed (i.e. used) over arelatively long period. Cans of food, for example,offeredcheaply fora limitedperiod,maybestoredat home without deterioration for months andconsumedattheusualfrequency.

1.23 Another issue concerns the definition ofusage.Abottleofmilkwilltypicallybeconsumedwithinafewdaysofpurchase.Consumptionislikelyto take place in the month for which the CPI iscalculated.Butasemidurablesuchasashirtwillbewornmany timesoveraperiodofprobablyseveralyears.Adurablesuchasatelevisionsetmaybeusedfor a decade or more and a house over an evenlonger period of time. The question arises as towhich CPImonth (ormonths) should the purchasebeallocated.

1.24 With services these questions can be evenmorecomplex.Take,forexample,thepurchaseofaseasonticketforabusservice.Thismaybeasinglepaymentforapasswhichgivesfreebustransportforayear.Itcanbeseenthatalthoughthisexampleis clearlya service (theuseofbus transportoveraperiod of time) it has much in common with thepurchase of a durable such as a television or shirtwhichprovidesatypeofserviceovera longperiod.A service such as amedical operation can also beregarded as durable, since it is likely to give longtermhealthbenefitstothepatient.

1.25 CPI theory devotes much thought to theseissues, which can have important implications notonly for how a CPI is compiled but for the resultsthemselves. Three different approaches can beidentified:

The acquisitions approach relates towhen thegoodorserviceisacquired,irrespectiveofwhenit is actually used or consumed. The time ofacquisition of a good is themoment atwhichthe legal ownership of the good passes to theconsumer.Thisisusuallythepointatwhichthepurchaserincursaliabilitytopay.Ontheotherhand, with a service there is no change inownership; it is acquired at the time theproducerprovidesit(e.g.thebusjourney).ACPI

basedonthisapproachmeasuresthechangeinthe cost of acquiring a product. The timing ofthe recorded prices should be consistentwiththewayinwhichthevaluewouldberecordedintheexpendituredatausedfortheCPIweights.

The use approach relates to the period overwhich theproduct is consumed or used; a CPIbasedonthisapproachmeasuresthechangeinthecostofusingtheproductovertime;inotherwordsthecostofthegoodisdistributedoveritsuseful life. Expenditures on durable goods andservicesareliabletofluctuate.

Thepaymentsapproachrelatestotheperiodoftime when the actual periodtoperiodpayments for the product aremade. This candiffer from theperiodwhen it is acquired andwhen it isused.Whenpaymentsarenotmadeincash, theremaybea longperiodbefore thepurchase ispaid for,whetherbycheque,creditcardorothermethod.The timeatwhichthesedebitsaremadeisirrelevantfortherecordingoftheprice.Theprice tobe recorded is thepricepayable at the time of acquisition (thoughsometimes themethod of paymentmay itselfaffecttheprice).

1.26 Statisticalofficesneed tohavea clearpolicyonwhichof these approaches isused in itsCPI. Inpractice,thechoicebetweenthethreeapproachesisan issue relating todurablegoodsand its impact islikely to be limited to theweight given to owneroccupier housing costs. The latter is discussed inSection 9.1 of the Handbook. In countries wherefood expenditures and other expenditures on nondurables, semidurables and even services accountfora significant shareof theCPIbasketandwherecredit financing is rarely used, the acquisition, useand payment approaches will give very similarresults and hence the CPI can satisfy many usesequallywell. This is the principal reasonwhymostcountries use, either implicitly or explicitly, theacquisitions approach to define what constitutesconsumptionexpenditure.

ContentsoftheHandbook

1.27 The compilation of CPIs is highly complex.Readerswillnotethat itoccupieseighteenchaptersoftheHandbook.Althougheachchapterdealswithaparticular topic, it is inevitable in sucha complexprocess that there are many interdependenciesbetween the topics. It is not possible to makedecisions on sources of weights data without firstdeciding on such factors as geographical andpopulation coverage, for example. So there aremanycrossreferenceswithineachchapter.

INTRODUCTION

5

1.28 The following summary of the contents ofeach chapterdrawsattention to someof themorecruciallinksbetweenchapters.

Chapter2onthescopeof theCPI,coverssuchissues as: the different uses of CPIs;geographical coverage (including regionalcoverage);thereferencepopulation;thechoicebetween democratic and plutocratic weights;the exclusions of certain products such asnarcotics; and the treatment of imputedtransactionsandimputedprices.

Chapter 3 deals with the subject of productclassification,andfocusesalmostentirelyonthestandardinternationalclassificationCOICOP.

Chapter4 isdevoted to the important issueofexpenditure weights. There are links to issuesdealt with in other chapters, including:plutocratic and democratic weights; theacquisition, use or payments approach to therecordingofprices; the treatmentof taxesandsubsidies; the population and geographiccoverage; the treatment of ownaccountproduction; the treatment of inkindremuneration;andthecoverageofsecondhandgoods. All of these topics are dealt with ingreater detail in other chapters. Chapter 4covers in depth the sources of weights data,including Household Budget Surveys andadjustmentswhichmaybeneeded;theweightsreference period and priceupdating; and theuseofweightsincalculatingtheoverallindex.

Chapter 5 deals with the complex issue ofsampling. CPIs necessarily rely on the use ofsamples covering time, location, outlettype,product type, and detailed productspecifications. All of these aspects of samplingaredealtwithinChapter5.

Chapter 6 on price collection, follows logicallyfrom Chapter 5. It deals with most of thepractical aspects of price collection, including:frequency and timing of collection; period orpointintime pricing; outlet types; seasonalavailability (alsodealtwith inChapter9);pricevolatility; the management of price collectionincluding quality control and documentation;staff management and training; methods ofrecordingprices;treatmentofspecialoffersanddiscounts;bargainedprices;centralcollection.

Chapter7dealswiththeproblemswhichoccurwhen products or outlets become unavailable,eithertemporarilyorpermanently.Thechaptercovers the important questions of substitution(replacement)andqualityadjustment.

Chapter 8 covers the treatment of newlyavailableproducts.Itdealswithplanningfortheintroductionofnewproducts;thetimingoftheirintroductionintheCPI;samplerotation;howtoadd a new elementary aggregate; and theconsequential effects on the calculation of theoverallindex.

Chapter9dealswithavarietyofspecificgoodsandserviceswhichneedspecialtreatment.Theyare:

o Owneroccupied housing the variousmethodsofattributingvaluestothesheltercostsofowneroccupiers.

o Ownaccount production, with particularemphasisonhomeproducedfood.

o Certain services, in particular health andeducational services, where there areusually issues concerned with free orhighlysubsidisedprovision.

o Tariffs suchas thoseused inutilities likepublic transport, gas, electricity andwatersupply,etc.

o Seasonal products methods of dealingwith products (especially fresh foodproducts) which are available in certainmonthsbutnotinothers.

o Secondhand goods, such as clothing andmotor cars. The chapter covers situationswhere secondhand items are traded viadealersordirectlybetweenhouseholds,anddistinguishes between imported anddomesticproducts.

Chapter10 isdevoted to thedetailedmethodsof calculationof CPIs, focussing in turn on thecompilation of elementary aggregate indicesand then the stagedaggregation to theoverallCPI. Itcoverssuch issuesasthestratificationofelementary aggregates; the use of implicitweightswithinelementaryaggregates;theprosand cons of various elementary aggregateformulae; the choiceof formula for calculatingtheaggregateindices;andthechainlinkingandsplicingofindices.

Chapter11dealswiththecalculationofvariantsofthestandardCPI,including:indicesrelatingtoparticularsocioeconomicgroups;regionalCPIs;indicesexcludingcertainproductgroupssuchasalcohol and tobacco; constanttax indices; andseasonaladjustmentoftheoverallCPI.

Chapter 12 deals with data validation at allstages of compilation of a CPI. This includeschecking at the initial input stage; credibility

PRACTICALGUIDETOPRODUCINGCONSUMERPRICEINDICES

6

checking;theauditingofcompilationprocesses;outputediting of results; the treatment ofoutliers; data editing, including the use ofalgorithmssuchastheTukeyalgorithm.

Chapter13dealswiththesourcesoferrorsandbias,includingadviceonhowtominimisethem.It deals specifically with questions of bias inelementary aggregates and upper levelsubstitution bias; quality change and newproducts bias; new outlet bias; and differentsourcesofsamplingandnonsamplingerror.

Chapter 14 deals with issues concerning thepublication of CPIs, including analysis andpresentation.Specificpointscoveredare:timingofpublication;prereleaseannouncementsandaccess; associated political statements;examplesofdetailedproceduralarrangements;press releases; revisions; special presentationsandanalyses.

Chapter15addressesawiderangeof issuestodo with the organisation andmanagement ofthewhole CPI production process. It describestypicalqualitymanagementprocesses; internalauditing; review systems; and risk assessmentanddisasterplanning.

Chapter 16 is concerned with CPI userconsultation, focussing inparticularon theuseofCPIadvisorycommittees.

Chapter17addressesthesubjectof integrationof the CPI production process with the ICP(International Comparison Program forPurchasing Power Parities). It examines thedifferences and similarities between the twoprogramsonespatial,theothertemporal,andsuggests ways of achieving maximum overlapand hence efficiency benefits. It also discussesways in which both processes can achievemutualstatisticalimprovementsintheirresults.

Chapter 18 provides guidance on reportingmechanisms forprovidinguserswith themetadata that they are entitled to on thecharacteristics, quality, access and integrity ofthe CPI and, correspondingly, the informationthat a national statistical institute needs tomonitor inordertoknowwhether it ismeetinguserneedsandtoput inplaceaprogrammeof

improvements. In particular, the chapter looksattheuseofqualityframeworksandchecklists.

TerminologyintheHandbook

1.29 Thefieldofconsumerprice indicesusesverylittle terminology specific to the subject. It usesmany terms in general use in statistics andeconomics,suchassample,indexandweight.Italsousesmanywordsandexpressionswhichareused in common language: examples includeproduct, item and basket. Thesewords canhave very specific meanings in a CPI context. Inrecentyearstherehasbeenamarkedincreaseinthedegreeof internationaldiscussionandcooperationamongstCPIexperts,andthishashadthefortunateresult of bringing about greater harmonisation ofterminology. It has also drawn attention to theproblems of translation, especially amongst thesecond groupof commonlyusedwordsmentionedabove.

1.30 The CPIManual includes a lengthy glossarywhich it is to be hoped will assist in the gradualharmonisation of terminology. This Handbook haswhereverpossibleusedthesametermsasarefoundin the Manuals glossary. There is, however, aseparate glossary so that readers do not need toconsult the glossary in the Manual. Most of thedefinitions are identical, but in some cases thewordinghasbeenchangedto improveclarity.Somenew words and phrases have been added, whilesome not referred to in the Handbook have beenomitted.

1.31 Care has been taken to use throughout theHandbookthesamewordsandphraseswheretheserefertothesameobject.Thishasbeendoneforthesakeof consistency, since it couldbemisleading tothe reader to use different terms for the sameobject.AnexampleistheuseofHouseholdBudgetSurvey for the type of household survey whichrequires households to state their expenditures onvarious products or groups of products. Countriesoften use a different name for their equivalentsurvey, but reference to the glossary will explainwhat is intended. Consistency has also called for auniformtitletobegiventosuchbodiesasNationalStatistical Institutes(NSIs),regardlessoftheiractualnames,whichvaryconsiderably.

INTRODUCTION

7

Appendix1.1 StageofProcessingFramework

/afromwholesalers/dealers/importagents

/bcapitaleqptpurchasedthroughwholesalers/agentsarenotcoveredbytheIIPwhichreflectschanginglevelsofoutputprices(iemanufacturerslistororderprices)

/ctheremaybedirectpurchases,butsuchtransactionsarenotreflectedinanypublishedindices

/dTheonlyexportpricescollectedbyPPSarefromtheproducers(EPI1s).TheEPI2spublishedbyTradeStatscoverallexportedgoods;butonlytheEPIsdeterminedbyPPSarebasedondirectpricecollection

/ethisblockisequivalenttothesaleoflabourbyprivatehouseholdstotheproductivesectorsoftheeconomy

/fmostcapitalgoodswill,bydefinition,feedintointermediatedemandbutcapitalgoodspurchasedforresalebydealers(egcars)willbeclassifiedasintermediatedemand

/gnotyetbasedondirectlycollectedpricesbutanestimateofimportedcapitalgoodspricesismadefortheFEPI(andforthePINCCA)

/hexpenditureonservicesincurredasanintegralpartoftheacquisitionofcapitalgoodsisclassifiedaspartofinvestmentexpenditure

/ioutputfromtheconstructionsectorfeedsintoGovtfinalconsumption(why?)

/jfromwholesaletoretail

/kegdomestichelp

/lcapitalgoodsthatfeedintoGovtFinalconsumption(egPCscosting

8

CHAPTER2

TheScopeoftheCPI Introduction

2.1 Chapter 1 of the CPI Manual provides asummary of the background to CPI methodology.Paragraphs1.8to1.12providesomebackgroundontheoriginsanduseofCPIs,includingtheirevolutionfrom compensation indices to macroeconomicindicesusedbygovernment toset inflation targets,and paragraphs 1.13 to 1.146 discuss thecharacteristics of the different index formulationsavailableandtheirdifferentproperties,includingthetests that can be applied in deciding whichformulation to adopt in different circumstances.Much of the remaining parts of Chapter 1 coverissuesrelatingtoconcepts,thescopeanddefinitionof a CPI and a wide range of other subjectsconfronting the index compiler including theacquisition, payments and usercost approaches,costofliving indices, the construction ofexpenditure weights and the collection of pricesdata. Chapter 2 of the CPI Manual discusses ingreater detail the different uses of a CPI and thepracticalconsequencesforthepricestatistician.

2.2 The ILO Resolution on CPIs specificallyaddressesthescopeoftheCPIinparagraphs8to14andmakes the point that the scope of the indexdependson themainuse forwhich it is intendedand should be defined in terms of the types ofhouseholds, geographical areas, and the types ofconsumergoodsandservicesacquired,usedorpaidforbythereferencepopulation.

2.3 ACPI canbeused foravarietyofpurposes,the more common ones being: the indexation ofwages,rents,contractsandsocialsecuritypayments;the deflation of household consumption in thenational accounts; as a general macroeconomicindicator, especially for inflation targeting and forsetting interest rates. Elements of a CPI are alsooften used in the calculation of purchasing powerparities (PPPs) required in the InternationalComparisonProgram(ICP)1.

2.4 GiventhemanyusesofCPIs,itisunlikelythatone index can perform equally satisfactorily in allapplications. Some countries therefore construct

1SeeChapter17.

severalCPIvariantsforspecificpurposes.Eachindexshould be properly defined and named to avoidconfusion and a headlineCPImeasure shouldbeexplicitlyidentified.WhereonlyoneCPIispublished,itisthemainusethatshoulddetermineitstypeandscope.Ifthereareseveralmajoruses,compromisesmay have to be made with regard to how it isconstructed.ThepurposeofaCPI should influenceallaspectsofitsconstruction.CPIproducersneedtoknow how their index is being used if they are toensure that it is fit for purpose. In this connectionuserconsultation is important.Chapter16providesdetailedadviceonconsultationprocedures.

2.5 Thischapterreviewstheissuesconfrontedbytheindexcompilerrelatingtothescopeoftheindexand the practical measurement and compilationdecisionswhichhave tobemade.Butbeforedoingso it reviews the various uses of a CPI to put thediscussionincontext.

ThedifferentusesofaCPI

2.6 CPIshavethreemainuses: Indexation

A CPI used forwage or contract indexation ofanyspecificgroup,whetherofpopulationorofproducts,should represent thecoverageof thegroupconcerned.Forinstance,itcanbearguedthattheweightsofaCPIusedfor indexationofpensions should cover only the expenditure ofthe pensioner population. The product andoutlet list could also be more appropriatelytargeted, if the data exist. This means, forexample,thata CPIusedfor indexingpensionsmay use weights relating to pensionerhouseholds and may exclude products whichmaybethoughtlargelyirrelevantto,say,poorerhouseholds, such as luxury items or brands.Similarly, for domestic indexation, the CPIshould cover only the expenditure of theresident population (see section below onGeographicalcoverage).Moregenerally,ithasto be decided whether the CPI should be, inprinciple, a costofliving index or a pure priceindex these two very different concepts arediscussedinChapter1.

THESCOPEOFTHECPI

9

For certain very specific types of indexation,such as for rents,usersmayprefer touse justthe subindex for rents. In suchcases, the subindexshouldbeofastatisticalqualitysufficientforthatpurpose.

NationalaccountsdeflationThis use requires consistency between thepricesdatausedfortheCPIandtheexpendituredata used in the national accounts. Both datasets should cover the same set of goods andservices and use the same concepts and sameclassification,whichfornationalaccountswouldbeCOICOP.Forexample,thenationalaccountsrequire the valuation of ownaccountconsumption, whereas this is often excludedfrom theCPIeither as amatterofprincipleorfor pragmatic reasons. This applies to thevaluation of the services of owneroccupiedhousing as well as the consumption of ownproducedfood.

InflationtargetingItcanbearguedthatcentralbanksideallyneedatimelyindexrelatingtototalinflation,notjustconsumer inflation. But NSIs generally areunabletoconstructsuchindices,inpartbecauseof the measurement issues relating togovernment consumption. In the absence ofsuchan indexmostcentralbanksrelyonaCPI,usingthedomesticconceptbutmeasuredonaswide a basis as possible, in terms of bothproductsandgeographicalcoverage. Thesameapplies to the use of the CPI as a generalmacroeconomicindicator.

2.7 The remainder of this chapter considers inmoredetail the issueswhich arise andprovides anintroduction to topicswhicharedealtwith inmoredetail in subsequent chapters. In particular, thereader is referred to Chapter 4 formore detailedadviceonthe issueofdemocraticversusplutocraticweights and on taking into account the nationalversus domestic concepts, institutional householdsandownaccountconsumptionintheconstructionofweights. Chapter 9 addresses in more detail theissueofsecondhandgoods.

Geographicalcoverage(References:CPIManual3.973.104;ILOResolution1012)

2.8 Therearetwodistinctaspectstothequestionof the geographical coverage of a CPI. The firstrelates to thecountryasawhole, thesecond to itsregions.

Thenationalversusdomesticconcept

2.9 ACPIcanhavenationalordomesticcoverage.

National coverage means that the CPI shouldcoverthenonbusinessexpenditure(andprices)relating to the resident population of thecountry, regardless of where the expendituretakesplace.Thenationalconceptisappropriatewhen the CPI is being used for indexation ofincomes. The weights for expenditure abroadcan be included in the Household BudgetSurvey,butmeasuringpricespaidabroadposesserious problems. The national concept thusposessomemajormeasurementproblems.

Domestic coveragemeans that the CPI shouldcover all the expenditure made within thedomestic territoryof thecountry, including thenonbusiness expenditure made by foreignvisitors. It isappropriatewhere theCPI isusedfor national inflation analysis and inflationtargeting.Many countries carry out surveys ofthe expenditures of foreign visitors, forexample, via International Passenger Surveysconducted at major border crossings andairports.This isparticularly important forthosecountrieswhichhavea largenumberofforeigntourists, or a high level of crossbordershoppers.

Foreignvisitorswillgenerallyhaveverydifferentexpenditure patterns from those of nationalresidents (e.g. theywill spendmore on hotelsand restaurants) and to omit them couldintroduceseriousdistortionsintoaCPIaimingtofollow the domestic concept, especially if themain purpose of the index is tomeasure theinflationarytrendsintheeconomy.

2.10 Purchasesmadeviatheinternetfromforeignwebsites or websites of retailers based abroadshouldbe included ifaCPI isbasedonthenationalconcept. So also should purchases made abroadmore generally, including such items as fees forforeign boarding schools, even if the item, in thiscase education, is consumed outside the country.Where such purchases are made in the foreigncurrency,theyshouldbeconvertedtothedomesticcurrency at the relevant exchange rate. Clearly itwould be impracticable to collect prices directly inforeign countries on a continuous basis althoughsurveys of prices done in order to computepurchasingpowerparitiesmayprovideanoccasionalbenchmark. Where the regular collection of therelevantpricesisnotpractical,itmaybepossibletoobtain a reasonable proxy for price movements

PRACTICALGUIDETOPRODUCINGCONSUMERPRICEINDICES

10

using published subindices of the other countriesCPIs.

2.11 Although no firm rules exist about thetreatment of internet shopping in respect to thedomestic concept, most countries which haveexamined the issue have concluded that internetshopping from domesticwebsites, and paid in thenationalcurrency,shouldbeincluded.

Regionalcoverage

2.12 As far as regional coverage of the CPI isconcerned, the general rule is that a national CPIshouldcoverexpendituresandpricesthroughoutthecountry. However, comprehensive coverage is notalwaysnecessary,especially if regionalCPIsarenotpublishedandthesamplingschemeensuresthattheindexisrepresentativeofthewholecountry.Insuchsituations, CPI compilers should collect evidencefromtimetotimeonthetrendsinpricesindifferentregionsoverperiodsoftimewhichcoverdifferencesinseasonalvariationsetc,toensurethatthesampleremains representative. Any region which showsprice trends significantly different from the othersshouldbecoveredbytheCPI if its inclusion is likelytohavea significanteffecton thenationalCPIandwill improve representiveness. But there is littlepoint in spending scarce resources collectingpricesinsparselypopulatedregionsiftodosowouldhavelittle or no impact on the national CPI. Whencarrying out such sensitivity tests regionalweightscanoftenbe an issue. In this casepopulationmaysometimesbeusedasaproxyforregionalconsumerexpenditure. However, where regional CPIs areaggregated to compute the national CPI, weightsshouldbebasedonregionalexpenditureratherthanpopulationdata.

2.13 Clearly, if a country produces regional CPIs,eachregionwouldnormallybecovered,eventhosewith sparse populations. But the cost ofmountingsuchsurveysisoftenfelttobeabarriertoproducingsuchindices.

2.14 Another difficulty regarding regional CPIs isrelatedtothenational"versus"domesticconcept.Itcansometimesbethecasethatahousehold livesin one region but doesmost of its shopping in anadjacentregion,particularlywhenahousehold livesclose to a regional border. The question ofwhether the expenditure weights and the pricesshouldbeallocated to the regionofexpenditureorthe region of residence is usually dictated bypractical issues. As with the national conceptdiscussedabove,iftheregionofexpenditureisused(equivalent to expenditure abroad by a domesticresident)somemeanshastobefoundofestimatingthe proportions of expendituremade by visiting

consumersinthevariousregionssothatthiscanbereflectedintheprices.

2.15 Finally, the question often arises as towhether a CPI can be limited to urban areas or ifrural areas should also be covered. Again, inprinciplethewholeterritoryshouldbecovered,butclearly the impact on the national CPI of includingrural areas where relatively few monetarytransactionstakeplacewilloftenargueagainsttheirinclusionongroundsofcost.Buttheviewtakenwilldepend,atleastinpart,onthesizeandtreatmentofownaccount production. If ownaccountconsumption is included in the CPI, the weightsshould includeavaluationofthephysicalquantitiesof suchproducts, the latteroftenderived from theHousehold Budget Survey. The priceswill normallybethesameasthoseusedforactualtransactionsforthesamegoodssoldinthesamelocality.

2.16 WheretheweightsderivedfromaHouseholdBudgetSurveyareavailableforruralaswellasurbanhouseholds it isgenerallybetter touse theweightsfor urban and rural households combined, even ifpricecollection is limitedtourbanareas,asthiswillnormallyimprovetherepresentivenessoftheindex.But, where feasible, price statisticians shouldundertake pilot calculations to testwhether this isthecase.

ReferencepopulationfortheCPI(References:CPIManual3.903.96;ILOResolution1012)

2.17 Householdscompriseoneormoreindividualslivingtogetherandsharingresources.TheofficialUNdefinition also includes people living in militaryaccommodation, convalescent homes andboardingschools. Such people are treated as belonging toprivate households. But for the purposes of CPIconstructionconsiderationmustalsobegiventotheinclusion of other institutional households, such asthose living inreligious institutes,hospitals,prisons,retirement homes etc. Temporary foreign workersmay live together in special housing blocks,whichmay also be treated as institutional households inthe population census. Expenditure onaccommodationandlivingcosts,suchaslodgingfeesandchargesformeals,imposedbytheinstitution,aswellaspersonalexpenditurebytheindividualon,forexample,clothesandtoiletries,shouldbeincludedinthe CPI. However, care should be taken to ensurethere is no double countingwhere, for instance, afamily rather than an individual pays theaccommodation costs charged to a patient in ahospital. The Household Budget Survey should bedesigned to pick up the amount spent on such

THESCOPEOFTHECPI

11

charges justonceandthestandardconvention istorecord it against the household which incurs thecosts. In the above example, the accommodationcosts should be included under the expenditure ofthe family, and not the individual. If individualsspend their own money on clothes and otherincidental expenditure then the Household BudgetSurvey should record this expenditure as beingincurred by the individuals. In reality, manyHouseholdBudgetSurveysdonotcoverinstitutionalhouseholds and, where this expenditure isconsidered tobe significant,estimateswillneed tobe made from, for example, special surveys ofpeople living in institutions or by reference to theexpenditure patterns of similar people, say, thesame sex, age and socioeconomic group, living innoninstitutionalhouseholds.

2.18 In reality, in considering the practical issuesrelating to the inclusionof institutionalhouseholdsinaCPI,twoquestionsneedtobeasked.First,istheexpenditure pattern of institutional residents likelyto be significantly different from householdresidents?Secondly,eveniftheanswerisyes,wouldtheirexclusionfromtheCPIbe likelytosignificantlyaffectthenational(orregional)CPI?Toanswerthesequestions,someresearchshouldbecarriedoutonasamplebasis.

2.19 Some countries exclude certain householdtypesfromtheCPI,suchastheverywealthyorthevery poor. Such exclusions may be on theoreticalgrounds (forexample,using the argument that theexpenditureofthewealthy,whoarerelativelyfewinnumber,shouldnotbeallowedtoaffectaCPIwhichmay be used for indexation of wages of ordinaryworkers)oronpracticalgrounds(forexample,usingtheargumentthatwealthyhouseholdstendtohavelow response rates to Household Budget Surveys,and their inclusion can lower the quality ofexpenditure weights). For a CPI which is used forindexationofwages,theexclusionofpensionerandwealthyhouseholdsmaybe justifiedon conceptualgrounds. For example, it may be considered thatsuchhouseholdsare likelytospendtheirmoneyonatypicalthingsandincludingthemwoulddistorttherelevant overall average. It is also argued by somethattheinclusionofpensionerhouseholdsshouldbeexcludedinprinciplefromanindexusedfortheuprating of state pensions because of the circularityinvolved (the level of state pension influencesexpenditurepatternswhicharethenusedintheupratingcalculation)whilstotherswouldargue that itis logical that indexation should be based on anindex reflecting the expenditure of pensionerhouseholds and their specific inflationaryexperience. Note that if wealthy households areexcluded,theCPIbasketshouldnotincludeproducts

likely tobeboughtonlyby theexcludedgroup,norshould outlets specialising in such products beincluded in the sample. Conversely, if thewealthyare included, some luxury products and outletsshould also be included in the sample. For theanalysisofnationalinflation,itisconsideredthatthemorecomprehensivetheCPIthebetter.

Democraticversusplutocraticweights

2.20 A democratic CPI uses weights whichaverage unweighted expenditure proportions overthe whole population, rather than dividing totalaggregated expenditure by the population. Thelatter method gives more weight to the highspending households, and is referred to as aplutocratic index. It is argued that a democraticindex is more suitable for showing the impact ofinflationon the averagehouseholdbut in reality isvery rarely computed by NSIs. There is a generalconsensusthataplutocraticindexistheappropriateindextousefornationalaccountsdeflationorforageneralmeasureofinflation.

2.21 Many countries publish a range of CPIsrelatingtosubsectorsofthepopulationsuchas:allhouseholds, lowincome households, pensionerhouseholds etc but a CPI based on democraticweightsisveryrare.

Productexclusions(Refs:CPIManual3.393.73;3.1213.127;ILOResolution1314)

2.22 In its role as an indicator of total consumerinflation,theCPIshouldinprinciplecoveralltypesofgoods and services which are consumed in thenationalretailingmarket. Inpractice,sometypesofproductmay be excluded for policy reasonswhileotherexclusions areunavoidable inpractice.Thesemay include:goods sold illegally, suchasnarcotics;black market sales; gambling; and prostitution. Inmost of these cases, therewill be no expendituredata fromtheHouseholdBudgetSurveyornationalaccountsandpricesaredifficult ifnot impossibletocollect.Whilstmanywouldconsideritinappropriatetohavewages indexedtoaCPIwhich includes,say,illegal narcotics, it should be noted that thevaluationofthetotalityofthenarcoticsmarket,andindeed, the nonobserved economy in general, isnow a requirement for the SNA as far as GDPestimates are concerned. Thus, in principle,estimatesofweightsandpriceswillneedtobemadefor thepurposeofproducingdeflators, even if theexpenditure isnotcovered intheCPI.Solutionswill

PRACTICALGUIDETOPRODUCINGCONSUMERPRICEINDICES

12

need to be found to the practical measurementissues.Forinstance,ifaCPIcoversgambling,itisnotthe gross stakes which should be included in theweight, but the net stakes, which is broadlyequivalent to the margin taken by the gamblingoperator.Asthis isnot likelytobemeasurable,onesolution may be to distribute the weight forgamblingacrossotherclasses inCOICOPgroup09.4(Recreationalandsportingservices).

2.23 Thetreatmentofsecondhandgoods isoftenfound to be problematic. As far as transactionswithinthehouseholdsectorareconcerned,saleswillbalancepurchases (apart fromanydealer charges),so the effect on the CPI is close to zero, and theymaybeexcluded.Butinmanydevelopingcountriesthereare significant salesof imported secondhandgoods, such as cars and clothing. Where sales ofimportedsecondhandgoodsaresignificantrelativeto sales of new goods of the same product, suchsalesshouldbeincluded,bothinweightsandprices.

2.24 CPIcompilerssometimesfaceproposalsfromgovernmentsorpressure groups toexclude certaincategories of product for nonstatistical reasons. Acommonexampleisalcoholwhereinsomecountriesitsconsumptionisassociatedwithsocialstigmaoritcanonlybepurchasedillegally.Whileitisacceptableto produce a variant of the general CPI excludingsuch products, themain CPI should include them,wherepractical,toensurethatthe indexpresentsatrueandaccuratepictureofnationalinflation.

2.25 TheCPIManualandtheappropriatesectionsofthisHandbookcoverinsomedepththetreatmentof other excluded or partlyexcluded products,including: taxes and licenses, subscriptions,insurance, gambling, financial transactions, hirepurchaseandinterestpayments.

Imputedtransactionsandimputedprices(Refs:CPIManual3.743.89;3.1303.132;ILOResolution1718)

2.26 A distinction can usefully bemade betweenimputedtransactionsandactualtransactionswhereaprice is imputed.Inthesecondcategorywouldbe

a prescribed medicine provided free as part of anational health service. There is a transaction inthe sense that a product changes hands but atzeropricesothatitdoesntconstituteamonetarytransaction. The conventions for a CPI constructedforthepurposesof indexationorthemeasurementof inflation as amacroeconomic indicator excludefrom coverage this category of nonmonetarytransaction,sonopriceshouldbeimputed.

2.27 A CPI should measure the prices of finalconsumption by a household. In principle, the firstcategory, imputed transactions where householdsdonotincurafinancialliabilitybutbearthecostsofacquiringthegoodorserviceinanotherway,shouldbe included in aCPIwhereused forGDPdeflationand, inprinciple,canbe included inaCPIcompiledfor other purposes. Perhaps the most importantexample is the consumption of ownproducedproducts such as food. Here, there is no actualtransaction at all, and thus no price. If thetransaction is to be valued a price has to beimputed.Thiswouldusuallybedonebyreferencetoactualpurchasesofthesameproductin,say,nearbymarkets.Butevenwhenthisisdoneforthepurposeof estimating GDP (which values consumptionregardless of its market mechanism) it is notnecessarily appropriate to include it in the generalCPI or in a CPI used for indexation where thenarrowestconceptofconsumptionthatcanbeusedis one based onmonetary expenditures only. Boththe CPIManual and the ILO Resolution leave thislatterquestionopen,but, fromthepointofviewofmeasuring inflation and also for the purposes ofincomeindexation,themostcommonviewisthatitis best to omit it on pragmatic grounds althoughgoodsand servicespurchasedbyhouseholdswhicharethenusedasinputsintoownaccountproductionare normally treated as if they themselves wereconsumption goods and services, and shouldtherefore be included in the CPI. Some countriesmayfinditusefultoproducetwoversionsoftheCPI:one includingand theotherexcludingownaccountconsumption.

2.28 The treatment of owneroccupied housing,which involves the own consumption of housingservices,isdealtwithseparatelyinChapter9.

13

CHAPTER3

ProductClassification 3.1 Paragraphs3.144to3.168oftheCPIManual,dealwith the subject of classification systems. Theparagraphslistthecriteriaforestablishingarelevantclassification system for a CPI. These are that: theclassificationmustreflecteconomicreality:meettheneedsofusersofCPIsubindices;beunambiguouslymutuallyexclusiveandexhaustive(that is,providingfullcoverage).Theparagraphsthengoontoreviewthedifferent classification schemesavailable to theCPI compiler and describe in some detail theinternational standard classification of individualconsumption expenditures COICOP (Classificationof Individual Consumption according to Purpose).Paragraphs20 to22of the ILOResolution reiteratethe point about addressing the needs of users ofsubindicesandgoontostatethatfor internationalcomparisonstheclassificationused foraCPIshouldbe compatiblewith COICOP at least at its divisionlevel.

Introduction

3.2 The importance of the product classificationsystem cannot be overstated. Choosing aclassificationsystemisthefirststepincompilingtheCPI because its subaggregatesmust be defined insuchawaythattheexpenditureweightsandpriceswill relate precisely to the coverage of the subaggregates.

3.3 Theclassification is importantalsobecause itestablishesaframeworkfromwhoseboundariestherepresentative items for inclusion in the index (andsometimestheoutlets)willbedefinedanddrawn.

3.4 Finally, the classification system helps indefiningwhichlevelofthehierarchywillbesuitableforpublication.

3.5 Classification is a central theme in thecompilationoftheCPI.Inyearspast,countriesusedtheirowndistinctsystemsforclassifyingtherangeofproductscoveredby theirCPI.Mostcountrieshavenow,however,movedtothe internationalstandardclassificationCOICOP seeabove.This chapterwilltherefore draw upon COICOP unless otherwisenoted.

3.6 It should be emphasised that the chaptershouldbenefitCPIcompilers intwoways evenfor

those that have not adopted COICOP. First, thechaptermay help in understandingmore fully theadvantages of COICOP and provide some guidanceformoving toaCOICOP structure if this isa futureobjective.Second,someoftheissuesthatareraisedin thischapterwillbe relevant formanyaspectsofCPIclassificationregardlessoftheoneactuallyused.

3.7 COICOPwasfirstdevelopedfortheSystemofNational Accounts (SNA 1993) to provide thestructure for classifying household consumptionexpenditure. Expenditures on the variouscomponents of household consumption are oftenused as the basis for the weights in the CPI (seeChapter 10). The 2003 ILO Resolution on CPIsrequires that national CPI classifications should bereconcilable with COICOP at least at its higheraggregation levels. Many countries have adoptedCOICOP in theireconomicstatistics (forexample, intheCPI,nationalaccounts,InternationalComparisonProgramme (ICP), and household budget surveys),with a clear advantage for integration of datasetsandenhancedanalyticalcapabilities.

Classificationsystems:thegeneralcase

3.8 In its broadest sense a classification is aprocedure in which individual items are organisedintocategoriesbasedoninformationononeormorecharacteristicsinherenttotheitems.Aclassificationscheme will usually have these same items (orelements)arrangedinahierarchicalorderedsystembasedon categorysubcategory relationshipswherethe subcategory has the same description as theassociated class in addition to one or moredescriptions.Forexample,anapple isa subclassoffruit.Soanyappleisafruit,butnoteveryfruitisanapple. A product needs to have a more detaileddescriptiontobeanapplethantobeafruit.

3.9 In principle, a classification system can bebased on any attribute of the objects beingclassified.Normally,organisingapopulationofitemsinto categoriesmust leave no two categorieswithany item incommon