Embed Size (px)
Citation preview
Measuring Employment OutcomesFebruary 5th, 2015
Agenda for Today
• Introductions
• Purpose of the event
• An explanation and summary of the work
• Presentations
• Q&A in response to presentations
• Where we go from here, Formation of the working group
• Suggestions
• Thanks and closing
Purpose of the Event
How can the identification and application of
appropriate indicators improve data-based decision
making for workforce development interventions?
• Frame the current issues around measuring
employment/labor market outcomes in the context
of workforce development.
• Create an open, inclusive space for all interested
stakeholders
• Support the Community of Practice to:
– Addressing this question
– Identifying the ‘must haves’ for measurement
– Adopting and harmonizing usage
A participatory process generates questions
Community of Practice Dynamics
Disseminating what is
learned through the
process
Prioritize issues, highlighting those
where more research is needed
Learning grants
Knowledge
products
Active participation
1.
2.
Talent cloud
activities
Training, capacity
building, internships,
field learning
3.
Frame the current issues
There is no global agreement or widely accepted best
practice governing the use of indicators to measure
outcomes in international workforce development
programming.
Great number of indicators in use, however cross-
comparable standard indicators are limited and there
is a reliance on project-by-project custom indicators.
There are successful measurement practices that have
been implemented from which lessons can be learned.
Literature Review Review of Literature
o Contextual Foundations - Methodologies, Frameworks, and Data Sources
o Field Analysis - Assessments, Evaluations, and Projects
Typologies
o Summary and Annotated Indicator Typologies
o Breakdown of Indicators (by indicator and typology)
o References Organization Source Outcomes Measurement
3 ConsortiumofStatesWashingtonWorkforceTrainingandEducationCoordinatingBoardOnbehalfof
theIntegrated
Performance
Information
ProjectState
Teamsfrom
Florida,
Michigan,
Montana,
Oregon,Texas,
&Washington
IntegratedPerformanceInformationforWorkforceDevelopment(February2005)
AccountabilityMeasures1) Labormarketresultsforprogram
participantsa) Dopeoplegetjobs?b) Whataretheypaid?
2) SkillsGaina) Towhatextentdoeducationlevels
increase?3) Resultsforemployersandtheeconomy
a) Arewemeetingtheneedsofemployers?
PerformanceIndicators4) Resultsforemployersandtheeconomy
a) Arewemeetingtheneedsofemployers?
5) Returnoninvestmenta) Whatisthereturnoninvestment?
AccountabilityMeasures1) Labormarketresultsforprogramparticipants
a) Short-termemprate:%employedafter2ndquarterb) Long-termemprate:%employedafter4thquarterc) Earningslevel:avg.earningsafter2ndquarterforthosewith
earnings2) SkillsGain
a) Credentialcompletionrate:%ofexiterswhocompleteindustry-recognizedcredentialw/in1year
3) ResultsforEmployersandEconomya) RepeatEmployerCustomers:%ofemployerswhoreturntothe
sameprogramforservicew/1year.PerformanceIndicators4) EmployerMarketPenetration
a)%ofallemployersservedin1year5) TaxpayerReturnonInvestment
a) Netimpactontaxrevenueandsocialwelfarepaymentscomparedtothecostoftheservices.
6) ParticipantReturnonInvestmenta) Netimpactonparticipantearningsandemployer-providedbenefits
comparedtocostoftheservices.
3 Additional
Information
ThisresourceisintendedforUSdomesticuse.Integratedperformanceinformation(IPI)reportsperformanceresultsconsistentlyacrossprograms,acrosslevels,orforprogramsasasystem.Itrespondstothelongstandingchallengeandfrustrationcausedbymultiple,inconsistentperformancemeasuresacrossWFDprograms,amultiplicitythatimpedescollaboration—inbothplanningandservicedelivery.Italsorespondstoshortcomingsinprograms'managementinformationsystemsthatcannotfollowparticipantsovertimeorreportperformanceinaconsistentmanner.IPIrequiresinstitutionsandpracticestosupportsharedaccountabilityforresults.
Whatmakesforagoodperformancemeasure(PM)?1) Areoutcomemeasures,notprocessoroutputmeasures2) Promotedesiredresults3) Areeasilyexplainabletoalayaudience(thefewerthemeasuresthebetter)4) Createalevelplayingfieldamongprogramsandservicestrategies(nobias)
Key Questions
• What indicators/frameworks are currently being used?
• What are the most common indicators and
measurement approaches?
• Are there identifiable trends and best practices?
Type of Indicator
Source Input Output Outcome System Level Outcome
Training USG Standard WFD Indicator Standard Foreign Assistance Master Indicator List
1. Number of days of USG funded technical assistance in workforce development provided to counterparts or stakeholders
1. Person hours of training completed in workforce development supported by USG assistance
Trends in measuring employment outcomes
o Need for improved M&E practices (using comparable
outcome indicators)
o Two main types of employment outcome indicators
o High degree of variability
o Employment status & wages are the most common WFD
outcome indicators
o High degree of divergence data gathering methodologies
o Building upon static indicators to form a more systems
approach
Randomized Control Trials, a
place in employment projects?
How can funders align?
Integrated Performance Indicators. Or, a Tale of Six States…
IPI Performance Measures
Accountability Measures Measures
Labor Market Results for Program Participants
Do people get jobs?
What are they paid?
Short-term Employment Rate:
The percentage of participants who are employed during the
second quarter after exit. (For youth, enrollment in education
counts as well as employment.)
Long-term Employment Rate:
The percentage of participants who are employed during the
fourth quarter after exit. (For youth, enrollment in education
counts as well as employment.)
Earnings Level:
Median earnings during the second quarter after exit among all
exiters with earnings
Skill Gains
To what extent do education levels increase?
Credential Completion Rate:
The percentage of exiters who have completed a certificate,
degree, diploma, licensure, or industry-recognized credential
during participation or within one year of exit
Results for Employers and the Economy
Are we meeting the needs of employers?
Repeat Employer Customers:
The percentage of employers who are served who return to the
same program of service within one year.
Performance Indicators
Accountability Measures Measures
Results for Employers and the Economy
Are we meeting the needs of employers?
Employers Market Penetration:
The percentage of all employers who are served during one
year.
Return on Investment
What is the return on the investment?
Taxpayer Return on Investment
The net impact on tax revenue and social welfare payments
compared to the cost of the services.
Participant Return on Investment:
The net impact on participant earnings and employer-provided
benefits compared to the cost of the services.
Source: Washington State Workforce Training and Education Coordinating Board. On behalf of the
Integrated Performance Information Project State Teams from Florida, Michigan, Montana, Oregon, Texas,
and Washington
Principal author: Bryan Wilson, Ph.D.
http://www.wtb.wa.gov/Activities_IPI.asp
Lara Goldmark | Project Director | FHI 360
[email protected] | +1.202.884.8392
Obed Diener | Technical Specialist | FHI 360
[email protected] | +1.202.464.3913
John Lindsay | Technical Specialist | FHI 360
[email protected] | +1.202.464. 3960
Eleanor Sohnen | Technical Advisor | FHI 360
[email protected] | +1.202.884. 8521
www.wfconnections.org
The project is funded by the USAID Office of
Education and managed by FHI 360, in partnership
with Child Trends, Making Cents International, and RTI International.
Rachel Blum | AOR | USAID
[email protected] | +1.202.712.4663
@wf_connections
Where do we go from here…
