MyChild Solution in Afghanistan - shifo.org · Shamati BHC (4) May 9th (1) May 9th (2) May 16th (3) May 16th (4) 10 Scanning MCS Senior Officer Mehterlam Liaison Office April 23rd

MyChild Solution in Afghanistan: An External Evaluation - Transfer of Work Processes to Existing Health System

Timothy Anderson

Tall inn University, 2018

2

Table of Contents

Glossary&Abbreviations.....................................................................................................................................3

Abstract .............................................................................................................................................................4

Introduction.........................................................................................................................................................5

Background..............................................................................................................................................................5

ResearchQuestions..................................................................................................................................................5

EvaluationStructureandFramework.......................................................................................................................5

Methods..............................................................................................................................................................9

StudySetting&Design.............................................................................................................................................9

StudyParticipants....................................................................................................................................................9

DataCollection.......................................................................................................................................................10

DataAnalysis..........................................................................................................................................................10EthicalConsiderations............................................................................................................................................11

Results...............................................................................................................................................................12

Conclusion.........................................................................................................................................................20

Limitations.........................................................................................................................................................20

References.........................................................................................................................................................21

3

Glossary & Abbreviations

Fixedsession:Animmunisationsessionthatoccurswithinahealthcentre/clinic.

Outreachsession:Animmunisationsessionthatoccursoutsideofahealthcentre/clinic.Duringoutreachsessions,healthworkerswillgenerallytraveltoruralorunderservedpartsofthedistrict.

EPI:TheExpandedProgrammeon Immunisationdevelopedby theWorldHealthOrganisation in 1974withthegoalofreducingchildmorbidityandmortalitybymakingimmunisationservicesavailableforallchildren.

EPIDashboard:Anonlineplatformwhereclinics reportdetailsand indicators fromtheir immunisationsessionsarevisualised.Allplanned,held,andmissedsessionsaredisplayedontheEPIDashboard.OnlydataforclinicsthatareusingMyChildSolutionisdisplayedintheEPIDashboard.

MCC:MyChildCard,thepaperhome-basedchildhealthcardusedwithinMyChildSolution.

SPF:Smart Paper Forms are the forms/vouchers are used to capture health information. Theseforms/voucherscanbescannedanddigitised.

SCA: The Swedish Committee for Afghanistan, a non-governmental organization responsible forhealthcareprovisioninLaghmanProvince,Afghanistan.

4

Abstract

Background:ShifohasrecentlyimplementedMyChildSolution,ahealthinformationmanagementsystemdesignedforchildhealth,withinMehterlamDistrictinAfghanistan.Thisreport,writtenbyanexternalevaluator,willprovideananalysisofwhich,andtowhatextent,workprocessesrelatedtoMyChildSolutionarebeingtransferredtospecificfacilityandmunicipallevelsofthehealthsystemstructuresinAfghanistan.

Objective:ThisevaluationfirstoutlinestheStandardOperatingProceduresneededforthetransferofMyChildSolutiontoMehterlamDistrict.Thisstudythenseekstodiscoverwhatessentialprocesseshavebeentransferred,andtoassesstheaccuracyandsustainabilityofthesetransferredprocesses.

Methods:Thisisaqualitativestudy,focusedonsemi-structuredinterviewsconductedwithhealthworkersatseveralclinicswithintheMehterlamDistrict.Interviewsareanalysedusinganarrativeanalysisapproach.Tosupportinterviews,researchaccesswasgrantedtosupportingdocumentsfromShifo.Thesedocumentsincludereceipts,printingorders,stockcards,datacompletenessreports,andverificationlogs.

Results:TheresultsofthisexternalevaluationindicatethatasignificantportionofprocessesessentialtoMyChildSolution-95%-havebeentransferredtothelocallevelinMehterlamDistrict.Inmostcaseswheretheseprocesseshavebeentransferred,theyarebeingperformedinawaythatisbothaccurateandsustainable.AllremainingprocessesareplannedtobefullytransferredtoMehterlambytheendof2018.

5

Introduction MyChild Solution, a health informationmanagement system designed for child health, developed byShifoFoundation,hasbeenimplementedandevaluatedinAfghanistan(MehterlamDistrict).Onecriticalstep to fully integratingMyChildSolutionwithin thecurrenthealthsystemstructure is the transferofessential tasks and routines to the local level. After training health workers in this new healthdelivery/data collection system, project’s goal is to gradually transfer processes connected with theinterventionandallowMyChildSolutiontobesustainableatthefacilityanddistrictlevels.Thisreport,writtenbyanexternalevaluator,willprovideananalysisofwhich,andtowhatextent,workprocessesrelated to MyChild Solution are being transferred to specific facility and district levelsof the healthsystemstructuresinAfghanistan.

Background MyChildSolutionwas introduced inAfghanistan inOctober2016withinMehterlamDistrict, located inLaghman Province. MyChild Solution was designed as a replacement for the previous healthmanagement information system (HMIS), whichwas primarily paper-based and involved a significantburden of administrative paperwork for health workers.MyChild Solution was intended to ease thisburden of paperwork, combining a paper child health card (MyChild Card) and a digital componentbasedonSmartPaperTechnology.

HealthworkersenterinformationabouttheservicesprovidedtoeachchildonSmartPapervouchersinMyChildCards,justasitwouldbenormallydonewithregularpaperusingballpointpens.Attheendofthe immunisation session, the nurse compiles all the vouchers from the day into a 'session bundle'.Followingthis, thevouchersarescannedatthescanningstationsituatedatMehterlamDistrict,whichhaselectricityandinternetconnectivity.Thedataisuploadedontoasecureserverwhereanelectronicmedicalrecordiscreatedforeachchild(Andersonetal2017).

While the introductionofMyChild Solutionhasbeen shown to reduce administration time for healthworkers,andhasreceivedpositivefeedback(seeAndersonetal2017), there isaneedtoensurethatthisnewhealthinformationmanagementsystemhasbeenimplementedproperlyandsustainably.

Research Questions 1.WhataretherequiredworkprocessesassociatedwithoperationsofMyChildSolutioninMehterlamDistrict(Afghanistan)?

2.InAfghanistan,howmanyoftherequiredworkprocessesforMyChildSolutionhavebeensuccessfullytransitionedtothelocalhealthsystemstructures?

3.Towhatdegreeare theseprocessesbeingperformed inamanner that is accurateand sustainablewithinthelocalhealthsystem?

Evaluation Structure and Framework Themajorprocessesandsub-processes listedherehavebeenadapted fromadocumentonStandardOperating Procedures provided by Shifo. The Standard Operating Procedures document details thestandardoperatingproceduresforMyChildSolution,includingalltheworkprocessesthatareessentialto the full functioning ofMyChild Solution. Each of these tasks has been categorized into five broad

6

types(ormajorprocesses).Thetablesbelowlistthemajorprocesses(andtheirrelevantsub-processes)thatneedtobeverifiedinAfghanistan.

1.Majorprocess:Procurementandsupplymanagement

Thismajorprocess involves thenegotiation,purchase,andconsistent supplyofallmaterialsused forMyChildSolution.

Table1

Sub-processes

ProcurementofMyChildCards/SmartPaperForms(MCC/SPF)

Purchasinginternetservices

Purchasingequipment(scanners)

SupplyandstockmanagementofMyChildCards(MCC)/SmartPaperForms(SPF)atdistrictlevel

SupplyandstockmanagementofMCC/SPFathealthcentrelevel

2.Majorprocess:Dataentry

Thismajorprocessinvolvesthedailyentryofpatient(child)dataontoMyChildCardsandSmartPaperForms,aswellastheirdeliverytoscanningstations.Dataentryismostoftendonebyavaccinator.

Table2

Sub-processes

DataentryusingMCC/SPF

DeliveringMCCVouchers/SPFtodesignatedscanningstations

Continuousfeedbackandimprovementsondataentrytools

7

3.Majorprocess:Scanningoperations

AfterdataentryandthedeliveryofSmartPaperForms,theFormsneedtobescannedandarchivedatscanningstationsatthedistrictlevel.

Table3

Sub-processes

ReceivingSmartPaperFormsandvouchersfromhealthcentres

ScanningandarchivingSmartPaperFormsandvouchers

Operationalmaintenanceofrequiredscanningstations

4.Majorprocess:Dataprocessing

Afterbeingscanned,patientdata fromSmartPaperForms isprocessedandsaveddigitallywithin thelocal health system.While someof thismajor process is automatic, there ismanualwork needed toverifyanydatathatcouldnotbeautomaticallyrecognisedduetopoorhandwriting.

Table4

Sub-processes

Automaticimageprocessingofhandwrittentextandcheckmarks

Dataverification

Reportingontechnicalerrorsandsystemimprovement

5.Majorprocess:Monitoringandevaluation

Thismajorprocess involvesmonitoringofdataqualityusingarangeof indicatorsandreportsthatareproduced by the system. This process ensures that reports are complete and timely. Any errors orinconsistenciescanbedetected,andfeedbackisprovidedtothehealthworkers.

8

Table5

Sub-processes

Ensuringdatacompleteness

Timelyreporting

Analysingdatarecordingerrors

Printingandhandingoverpaper-basedreportstohealthcentres

Findingpersonaldataofachildfromtheelectronicregisterwhenparentslosechildhealthcards

ProvidingfacilitybasedKeyPerformanceIndicatorsviaSMStohealthworkers

9

Methods Study Setting & Design This external evaluation is conducted for clinics inMehterlamDistrict, Afghanistan.MyChild Solutioncovers141healthservicedeliverypoints inMehterlam,amongwhich,thereare8fixedhealthservicedeliverypoints,105outreachhealthservicedeliverypointsand28mobilehealthservicedeliverypoints.

This is a qualitative study, focused on semi-structured interviews conducted with health workers atseveral clinics within theMehterlam District. To support interviews, research access was granted tosupportingdocumentsfromShifo.Thesedocumentsincludereceipts,printingorders,invoices,monthlyreporting forms, data completeness reports, and verification logs.In addition, supportingmedia files(videosandphotosofdailyoperationwithinclinics)wereprovidedbySCA.

Study Participants Thefirstpointofcontact isaMyChildSolution(MCS)SeniorOfficerworkinginSCAMehterlamLiaisonOffice.MCSSeniorOfficerresponsiblefor immunisation informationmanagementandcoordinationattheprovincial level for facilities thatuseMyChild system.Afteran initial SkypecallonApril23rd, theevaluator receivedcontactsof studyparticipantsthatcouldprovidedetailsofhoweachworkprocesshasbeentransferredandisbeingperformedatthelocallevel.Toprotecttheirprivacy,allparticipantsarereferredtobytheirtitlesratherthantheirnames.Table6belowprovidesalistofthesecontacts.

Allhealthworkersselectedforthisstudysatisfythefollowingcriteria:

§ They are recommended by an MCS Senior Officer as representative of MyChild SolutionprocesseswithinMehterlamDistrict;AND

§ TheyareresponsibleforMyChildSolutionand/orimmunisationsuppliesinMehterlamDistrict;OR

§ Theyhavebeenengagedinroutineimmunisationatahealthcentreforatleast1month

Table6

MajorProcess Relevant ContactPerson(s)

Clinicplacement Interviewdate

Procurement and supplymanagement

MCSSeniorOfficer Mehterlam LiaisonOffice

April23rd

Dataentry Vaccinator(1)

Vaccinator(2)

Vaccinator(3)

Vaccinator(4)

MehterlamCHC(1,2)

BabasahibBHC(3)

ShamatiBHC(4)

May9th(1)

May9th(2)

May16th(3)

May16th(4)

10

Scanning MCSSeniorOfficer Mehterlam LiaisonOffice

April23rd

Dataprocessing MCSSeniorOfficer Mehterlam LiaisonOffice

April23rd

Monitoring andevaluation

MCSSeniorOfficer Mehterlam LiaisonOffice

April23rd

Data Collection Giventhe inability to travel toAfghanistan (duetosecurity limitations) for interviewsandparticipant-observation, semi-structured interviews were conducted over Skype. Open-ended questions (seesamples below) were asked regarding each work process from the respective health workers, withspecific attention paid to where there have been inconsistencies or challenges encountered in thetransferprocessofMyChildSolution. In somecases, intervieweeswereasked tovisuallydemonstratehowparticularprocessesareperformed(e.g.,fordataentry).Allconversationswererecordedonvideotomaketranscriptionandsubsequentanalysismorereliable.Eachinterviewrangedfrom20minutesto70minutes,withanaveragetimeofapproximately40minutes.AllinterviewswereconductedinEnglishintheMehterlamDistrictOffice(wheretherewasthemostconsistentinternetconnectionavailable).

Open-endedinterviewquestionsthatwereaskedtoallparticipantsinclude:

Whatisyourrolewithintheclinic/officeandwithinMyChildSolution?

Howlonghaveyoubeenworkingwithinyourrole?

Whatdaily/weekly/monthlytasksareyouresponsiblefor?

Doyouhaveanyfeedbackontheprocessesyouworkwith?

HavetherebeenanyproblemsadaptingtoMyChildSolution?

Ifneeded,wouldyoubeabletoteachanewhealthworkerthestepsyouperform?

Data Analysis InterviewswereconstructedwiththeintentionofbothconfirmingtheprocessescarriedoutbystaffinMehterlam and discovering if these tasks could be passed on or taught to potential incomingstaff.Regardingtheresearchquestions,accurateandsustainableprocessesaredefinedasthefollowing:

1. Accuracy:Processesareperformed inamannerthatencompassesallstepsandsub-processesessential for the functioningofMyChild Solution (seeEvaluation Structure and Frameworkformoredetails).

2. Sustainability: Processes are well-understood by health workers and receive consistentfeedback.Stafffeelcomfortableperformingtheseprocessesontheirownandteachingthemtoothers.

11

Regardingaccuracy, the aim is to determine the extent to which each major process has beentransferredtothelocalhealthsystemstructureinMehterlam.Withintheresults,foreachmajorprocessand sub-process, a percentage is assigned. This percentage indicates whether the process has beenevaluated as fully transferred (100%), not transferred at all (0%), or partially transferred. Ideally, allprocesses under evaluation should be fully transferred toMehterlamDistrict; 100% is the goal in allcategories.

Regardingsustainability, staff were asked questions to ascertain how well they understand MyChildSolutionandhowcomfortabletheywouldfeelexplaining/teachingtheirroletoauditorsandnewstaff.

Interviews were analysed using a narrative analysis approach (see Chase 2005). Each interview wasviewedmultipletimes,astheevaluator lookedforthe‘story’ofthe interviewee’sdailyroutine.Thesenarratives, taken together, were used to give a holistic picture of howMyChild Solution operates inMehterlamonadailybasis.WithintheResults,theevaluatorhasaimedtorepresenttheintervieweeintheirownwords,usingthesameterminologythattheyemployintheirdescriptions.Timestampswereusedineachvideotocreatereferencepointsthatcouldbeeasilyreturnedtoduringtheanalysis.

Ethical Considerations Priortotheinterviews,allobjectivesofthisevaluationwereexplainedtoparticipants.Allintervieweeswereinformedthatparticipationinthisevaluationwasvoluntary.Theywerealsoassuredthattheyhadtherighttowithdraworrefusetoanswerquestionsatanystageoftheinterview.Recordedinterviews,transcriptions, andnoteswereonlyaccessible to theexternalevaluator. Toensure confidentiality,nospecific details of financial transactions (e.g., from receipts or order transcripts) have been includedhere.

12

Results Theresultsofthisevaluation,includingabreakdownofthetransitionstatusforeverymajorprocessandsubprocess,areavailableinaseriesoftablesbelow.Theevaluator'sunderstandingoftheaccuracyandsustainabilityofeachmajorprocessisalsoincludedinthetextbeloweachtable.Thedescriptionsofthesub-processesalsoserveasananswertothefirstresearchquestioninthistext:whataretherequiredworkprocessesassociatedwithoperationsofMyChild Solution inMehterlamDistrict (Afghanistan)?Amoregeneralsummaryoftheresultsisincludedattheendofthissection.

1. Procurement & Supply Management (100% transferred) Table7

Process Description TransitionStatus SupportingDocument(s)

Results

ProcurementofMCC/SPF

NegotiatingandcontractingprintingcompaniestosupplyMyChildCardsandSmartPaperFormstoclinics

Transferred(100%) ReceiptsfromtheKabulofficeoftheSCAconfirmingpurchaseofMCC/SPFthroughout2017

Supportingdocumentsindicatesuccessfultransfer

Purchasinginternetservices

Negotiatingandcontractinganinternetserviceproviderforscanningstations

Transferred(100%) AcontractbetweentheSCAandAfghanTelecomconfirminginternetservicesinMehterlamDistrictOffice

Supportingdocumentsindicatesuccessfultransfer

Purchasingreplacementequipment(scanners)

NegotiatingandcontractingSmartPaperscannersandreplacementrollers

N/A N/A Scannersareestimatedtolastfor7years,whilerollersneedtobereplacedannually.

SupplyandstockmanagementofMCC/SPFsatdistrictlevel

MonitoringstockandsecuringdeliveriesofMyChildCardsandSmartPaperformstoindividualclinics

Transferred(100%) ReceiptsfromtheKabulofficeoftheSCAconfirmingpurchaseanddeliveryofMCC/SPF

Supportingdocumentsandinterviewindicatesuccessfultransfer

13

throughout2017

SupplyandstockmanagementofMCC/SPFsathealthcentrelevel

MonitoringstockandsecuringdeliveriesofMyChildCardsandSmartPaperformstoindividualclinics

Transferred(100%) ReceiptsfromtheKabulofficeoftheSCAconfirmingpurchaseanddeliveryofMCC/SPFthroughout2017


Healthcare inMehterlamDistrict,Afghanistan is deliveredby the SwedishCommittee forAfghanistan(SCA), which has its head office in Kabul. The Kabul office is responsible for several aspects ofprocurement for Mehterlam, namely printing of MCC/SPF, distribution of MCC/SPF, and purchasinginternet services. TheMCS SeniorOfficer is an employeeof the SCA, primarily placed at the regional(Laghman) and district-level (SCA Mehterlam liaison/coordination) offices, where he is responsibleforimmunisation informationmanagement and coordination in facilities that useMyChild Solution attheProvinciallevel.

At the time, he is not facing any printing shortages, but the printing company tends to be latewithsupplies.Thecompanyoftenclaimstheywillsupplyitinonemonth,butoftendeliveriescantakeupto3months.However,evenwiththesedelays,stock isstill sufficient,andthecardsareprintedwell.HeputsvouchersandcardsinStockCards,suppliesaredistributedtohealthfacilitiesonceamonth.Healthfacilities are using them properly, and these facilities contact the MCS Senior Officer when moresuppliesareneeded.

2. Data entry (100% transferred)

Table8

Process Description Transitionstatus SupportingDocument(s)

Results

DataentryusingMCC/SPF

RecordingpersonaldetailsofchildrenontoMyChildCards;recordingdailyandmonthlyvaccineuse

Transferred(100%) Datacompletenessreportsgeneratedbythesystem;linkstoEPIDashboard


DeliveringMCCVouchers/SPFto

Bringingsmartpaperformsandvouchers

Transferred(100%) Datacompletenessreportsgenerated

Supportingdocumentsand

14

designatedscanningstations

forscanningtothescanningstationattheendofthemonth

bythesystem;linkstoEPIDashboard

interviewindicatesuccessfultransfer

Continuousfeedbackandimprovementsondataentrytools

SendingfeedbacktoShiforegardingtheuseofMyChildCardsandSmartPaperForms

Transferred(100%) N/A Interviewindicatessuccessfultransfer

Dataentry inMehterlam is completedby vaccinators. For this evaluation, interviewswere conductedwith4vaccinators fromthreedifferentclinics.Thesevaccinators fill in registration,givevaccines,andattendbothfixedandoutreachsessions.Vaccinatorsshowedhowtheyfilledouteachpaper(proofofID,registrationpaper,parents’names,dateofbirth,etc).Registrationvouchersarethenputinthedatabox and kept for one week. Feedback from the health workers is very positive; the burden ofadministration is greatly reduced in comparison to theprevioushealth information system. Individual(5-10min)andgroupcounselling(30min)tasksareperformed.

However, the vaccinators had also encountered several challenges within the data entry processes.Primarily,mothersaresometimesafraidtogivemobilenumbersorhusbands’workinformationoutforsecurity reasons. Sometimes parents also do not receive SMS due to poormobile network coverage.Anotherproblemisthat,becauseregistrationinformationisconfidentialandonlyavailabletotheMCSSeniorOfficer, patient data cannot always be retrieved by vaccinators themselves. If theMCS SeniorOfficerisnotintheoffice(e.g.,onSaturdays),thensometimespatientswhohavelostachildhealthcardcannotbehelped.This lackofcoordinationinworkinghoursbetweenvaccinatorsandtheMCSSeniorOfficer can occasionally be frustrating for the vaccinators. Often, mothers must travel a long waythroughdangerousareastogettotheclinic-itisthereforeveryproblematiciftheirvisittotheclinicisnotsuccessful.

3. Scanning operations (100% transferred)

Table9


Results

ReceivingSmartPaperFormsandvouchersfromhealthcentres

ReceivingandorganizingMyChildCardvouchersforscanning

Transferred(100%) DatacompletenessreportsgeneratedbyShifo;linkstoEPIDashboard

Interviewandsupportingdocumentsindicatesuccessful

15

transfer

ScanningandarchivingSmartPaperFormsandvouchers

RunningSmartPaperFormsthroughthescannerandensuringthattheirfilesarereceivedandprocessedbytheofficelaptop

Transferred(100%) Datacompletenessreportsgeneratedbythesystem;linkstoEPIDashboard

Interviewandsupportingdocumentsindicatesuccessfultransfer

Operationalmaintenanceofrequiredscanningstations

Cleaningandchangingscannerrollersonceayear;clearingpaperjams;ensuringlaptopfunctionality;internetmaintenance


The MCS Senior Officer is responsible for all scanning, and he receives session vouchers from eachfacility at the end of each week. Once a week, the MCS Senior Officer does voucher scanning,synchronizing,andverificationofalldata.Theprocessissosimplethathecancoverallhealthfacilitiesbyhimself.Afteroneweek,theyarewrappedwitharubberbandandtheMCSSeniorOfficercollectsthem.Hebringsthemintohisofficeandscansallofthem.

Regardingequipment,twoscannershavebeenreceivedfromtheMyChildproject.As longastheyaremaintainedproperly(e.g.,replacingrollersannually),thesescannersshouldworkforupto7years.Shifodeliversnewpartsforthescanners.

4. Data processing (83% transferred)

Table10


Results

Automaticimageprocessingofhandwrittentextandcheckmarks

Handledbysoftware(SmartPaperTechnologyEngine)

Transferred(automatic)(100%)

N/A ThisishandledautomaticallybySmartPaperTechnologyEngine

16

Dataverification Checkingandcorrectingforflaggederrors/marks/badhandwritinginvouchersscannedbySmartPaperTechnologyEngine.Thisinvolvestwosteps:primaryverificationandmasterverification

Partiallytransferred(50%)

VerificationlogsfromOctober2017-present

PrimarydataverificationisdonebytheMCSSeniorOfficer,whichwasconfirmedviainterviewandsupportingdocuments.However,masterverification(asecondstepwhereanyfurtherflaggederrorsarecorrected)isplannedtobetransferredbyendofyear2018.Atpresent,masterverificationisstillperformedbyShifostaff,inordertoanalyseerrorsandimprovemachinelearningalgorithmsandvalidationrules.

Reportingontechnicalerrorsandsystemimprovement

SendingfeedbacktoShiforegardinganytechnicalproblemswithsoftwareandsuggestionsforimprovement

Transferred(100%) N/A InterviewwiththeMCSSeniorOfficerindicatessuccessfultransfer

Afterscanning, theMCSSeniorOfficerperformstheprocessofprimarydataverification,checking fordataentryerrorsinMyChildCards.AnextensiveverificationlogfileconfirmsthattheMCSSeniorOfficerhasbeenprocessingdocumentsandverifying fields flagged for reviewsinceOctober2017.Hehasnocomplaintswiththeprocessandisveryfamiliarandcomfortablewithdataverification.Heisconfidenthecouldteachthesestepstoanyincominghealthworker.

17

TheonlycomplaintbroughtupbytheMCSSeniorOfficerwaswiththeslowinternetconnectioninhisdistrict office – it takes a long time for scanned vouchers to synchronize. Some tasks are delayed orencounterproblemsduetothispoorinternetconnectivity.Newinternetoptionswouldbeideal.

Dataverificationisonlyhalfwaytransferred(50%),sincethecrucialsecondstepofMasterVerificationisstilldonebyShifo.However,MasterVerificationisoncoursetobetransferredtoMehterlamDistrictbytheendofyear2018,atwhichpointdataverificationshouldbe100%transferred.

5. Monitoring and evaluation (90% transferred)

Table11


Results

Ensuringdatacompleteness

Measuredbycomparingthenumberofimmunisationsessions(bothfixedandoutreach)capturedinelectronicreportstothenumberofsessionsthatwereplannedtobeheldaccordingtofacilityschedules.Inthepossiblecaseofamissedsession,theresponsiblepersonmustcallthehealthfacilityandclarifywhetherthesessiontookplaceorwhyitwascancelledormissed.

Transferred(100%) AccesstoEPIDashboardanddatacompletenessreportsshowingthatmissingdataforfixed/outreachsessionsarebeingnotedandreported


Timelyreporting EnsuringthatallelectronicHMISreportsareavailabletothedistricthealthofficesbythe5thofeverymonth.TheEPIDashboardandemail

Transferred(100%)

LogsconfirmingthatreportsfromMehterlamhavebeenreportedontime

18

areusedasplatformsforsharingreports.

Analysingdatarecordingerrors

Ensuringthatanyrecordingerrorsmadebyhealthworkersareanalysedandthatanyrelatedfeedbackispassedontohealthworkers

Transferred(100%)

Accesstoanoperationsissueregisterwhichconfirmsthatrecordingerrorsarebeingloggedandanalysed


Printingandhandingoverpaper-basedreportstohealthcentres

Afterelectronicreportsaregenerated,thereportsareprintedandsharedwithfacilitiesthatdonothaveaccesstoelectronicreports


Extractingpersonaldataofaclientfromelectronicregisterwhenparentslosechildhealthcards

Properlyretrievinganyneededpersonaldatafromtheclinic'ssystemwhenaparent'sMyChildCardhasbeenlost


ProvidingfacilitybasedKeyPerformanceIndicatorsviaSMStohealthworkers

ThesystemcangeneratenewKeyPerformanceIndicators(KPIs)thatfacilitatemonitoringandevaluationofEPIperformanceatthefacilitylevel.KPIsaresentbySMStohealthworkersmonthly.KPIsareplottedandfolloweduponthemonitoringboardsbyhealthworkerseverymonth.

Partiallytransferred(38%)

N/A InterviewdataandcommunicationwithShifoindicatethatthissubprocessiscurrentlybeingpilotedin2clinicsoutofthe8fixedfacilities(38%)totallyoperatinginMehterlamDistrict

19

TheMCSSeniorOfficergoestohealthfacilitieseverymonthtosupervisetheMyChildworkprocess.Hesharesactivities,teachesthemhowtofillincards,detachvouchers,howtoadministervaccines,etc.Heis very comfortable training new staff. The MCS Senior Officer also checks monthly activity reports,registry reports, follow-up lists, and tally reports for any discrepancies. Any findings are sharedwithhealthworkers.

In addition to these steps, the MCS Senior Officer analyses monthly reports, checks coverage of allvaccines,andevaluateswastage.Someissuesarerelatedtohealthworkererrorandothers(rarely)tosystemerror.SystemerrorsaresharedwithShifo.TheMCSSeniorOfficerandShifoare incontactatleastonceaweektodiscussfeedbackandanyproblemsthatmayhavecomeup.

Otherproblemsaresharedwiththehealthworkersthemselves.Otherwise,allactivitiesarerecordedinEPI Dashboard. All fixed and outreach sessions conducted per week are recorded. Any missed orcancelledsessionsarementioned.Theseprocessesareallgoingwell.Itisasignificantimprovementtotheprevioussystem,wheremonthlyevaluationwasmuchmoretediousandtime-consumingandhadtobedonemanually.Allreportscannowbegeneratedelectronically.

Overall Transfer In total, this study estimates that 95% of processes essential for the proper functioning of MyChildSolution have been transferred to the existing health system structure in Mehterlam District inAfghanistan(seeTable12).

Table12

MajorProcess PercentageTransferred

Procurement&supplymanagement 100%

Dataentry 100%

Scanningoperations 100%

Dataprocessing 83%

Monitoring&evaluation 90%

TotalforMyChildSolution 95%

20

Conclusion The results of this external evaluation indicate that a significant portion of processes essential toMyChildSolution -95%-havebeentransferredtothe local level inMehterlamDistrict. Inmostcaseswheretheseprocesseshavebeentransferred,theyarebeingperformedinawaythatisbothaccurateand sustainable. All interviewees stressed that theywere comfortable in their roles, appreciated theeaseandconvenienceofworkingwithMyChildSolution,andexpressedtheirabilitytoteachincomingstaff these processes if needed. Only two sub-processes -Data verification and Providing KeyPerformanceIndicators-werenotentirelytransferredtoMehterlamDistrict.AllthesesubprocessesareplannedtobefullytransferredtolocallevelinMehterlamDistrictbytheendof2018.

Limitations Despite the generally positive trend of these findings, there are several limitations of this study thatshouldbeconsideredbyreaders.Theseare:

1. Duetosecurityconstraints,all interviewswereconductedthroughSkypeandSlack.Theabilitytoobserveworkprocesses inactionwasgreatlydiminishedoveravideocall (ascompared tobeingon-site),anditwasoccasionallydifficulttogetasenseofhowMyChildSolutionhadbeenintegratedwithinday-to-dayactionsandroutines.

2. Challengesincommunicationandtranslation.AlthoughallinterviewswereconductedinEnglish,in most cases, English was not the native language of interviewees. In all interviews withvaccinators, theMCS Senior Officer had to act as a translator. This opens the possibility formisunderstandingsandmistranslationsthatmighthaveaffectedresults.

3. The lownumberof interviewees.Outof 8 fixed facilities inMehterlamDistrict usingMyChildSolution,theevaluatorwasonlyableto interviewhealthworkersfrom3fixedfacilities.Whilenot ideal, time constraintsmade it difficult to conduct amore expansive study and acquire amorerepresentativesampleofhealthworkersfromacrossMehterlam.

Theselimitationshighlighttheneedforafurtheron-siteevaluationofworkprocesstransitionsatalaterdatein2020or2021,whenthesolutionisscaleduptothewholeLaghmanProvince.

21

References Chase,SE.(2005)."NarrativeInquiry:Multiplelenses,approachesandvoices."InNKDenzin&YSLincoln(Eds)The SageHandbookofQualitative Research(3rd Edition) pp. 651-679. ThousandOaks, CA: SagePublications.

Anderson,T.,Eshonkhojaeva,S.,Safi,H.,Hamidi,S.,Mikaelsson,M.(2017).“ImprovingChildHealthcarein Low Resource Settings: A Pilot Study of MyChild Solution in Afghanistan”. Shifo.https://shifo.org/doc/MyChildEfficiencyEvaluationAfghanistan.pdf/

Documents

MyChild Solution in Afghanistan - shifo.org · Shamati BHC (4) May 9th (1) May 9th (2) May 16th (3) May 16th (4) 10 Scanning MCS Senior Officer Mehterlam Liaison Office April 23rd