Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
O*NET: A Source of Information for Measurement ofInformation for Measurement of Socioeconomic Status?
Tom PlewesTom PlewesSenior Program OfficerCommittee on National StatisticsDivision of Behavioral and Social Sciences and Education
March 8, 2012
Discussion Topics
What is O*NET?
National Research Council Study
O*NET Strengths and Weaknesses as a Source of Socioeconomic Data
Selected Uses of O*NET Data for Socioeconomic AnalysisSocioeconomic Analysis
What is the Occupational Information Network (O*NET)?( )
Launched in 1998 by DoL to replace the Dictionary of Occupational Titles
Pro ides information abo t occ pation and orker characteristics for aProvides information about occupation and worker characteristics for a variety of purposes
– Career guidance– Reemployment– Counseling– Counseling– Workforce development– Research
Two componentsTwo components– Content model – frame work for occupational data– Searchable electronic data base
• www.onetonline.org
Content Model
Data CollectionContinuing data collection program populates and maintainsContinuing data collection program populates and maintains
the O*NET database.
Data collected by the National Center for O*NET Development, y p ,through its contractor Research Triangle Institute
Information is collected using a two-stage design in which: t ti ti ll d l f b i t d t-- a statistically random sample of businesses expected to
employ workers in the targeted occupations are identified and-- a random sample of workers in those occupations within
those businesses is selected and administered one of four standardized questionnaires.
Data CollectionSeveral hundred rating scales are provided based onSeveral hundred rating scales are provided, based on
responses by the sampled workers on the O*NET questionnaires.
Four questionnaires, each containing a different set of questions. Sampled job incumbents for each occupation are randomly assigned one of the four questionnaires.
All respondents are also asked to complete a task questionnaire and provide some general demographic information.
A fifth questionnaire, focusing on Abilities, is completed by occupational analysts (SMEs) using the updated information from incumbent workers.
Sample Questions
32. Static Strength The ability to exert maximum muscle force to lift, push, pull, or carry objects.
1 2 3 4 5
NotImportant* Important
VeryImportant
ExtremelyImportant
SomewhatImportant
A. How important is STATIC STRENGTH to the performance of your current job?
1 2 3 4 5 * If you marked Not Important, skip LEVEL below and go on to the next activity. B. What level of STATIC STRENGTH is needed to perform your current job?
Push an empty shopping cart
Pull a 40-pound sackof fertilizer across the lawn
Lift 75-pound bagsof cement onto a truck
1 2 3 4 5 6 7Highest Level
National Research Council Study Panel Charge
Document and evaluate current and potential uses of O*NET in: kf d l– workforce development
– HRM– disability determination
research– research
Explore linkage to SOC and other data sets
Identify improvements, especially in:– currency– efficiencyy– cost-effectiveness– use of new technology– content model
StrengthsTheoretically informed and research driven content model and comprehensiveTheoretically informed and research-driven content model and comprehensive
work descriptor taxonomies
Provides a “common language” across jobs.
Hierarchical organization of descriptors.
Both work activities and worker requirements (describes jobs and people).
Multiple types of descriptors and multiple levels of analysis (i.e., “multiple windows” of accessing job information) allows versatility of application.
Work descriptor ratings collected via differential use of both professional p g panalysts (for Abilities and Skills ratings) and job incumbents (for Knowledge, GWA, Work Styles, Education & Training, Work Context, and Task ratings) as appropriate for different domains.
StrengthsRigorous sampling plan, high response rates, strong business participation,
and frequent updates
Database is national in scope and maximizes data quality, currency, and li bilitreliability.
Provides of “metadata” (i.e., detailed rating-level, occupational-level, and distributional statistics) to users, enhancing data interpretation.
Provides mechanisms to collect data on new and emerging occupations, and to collect new task data on existing occupations.
Methodology allows for occupational description beyond the most detailedMethodology allows for occupational description beyond the most detailed occupational level of the SOC.
Data collection process and methodology vetted through rigorous OMB clearance process
StrengthsE f d f i i i b dEase of system access and cost-free use; minimizes burden
to public.
Standardized information within organizations and acrossStandardized information within organizations and across organizations and within industries.
Reduced costs of information gathering (than if done i di id ll /i d d tl )individually/independently).
Data/information availability and comprehensiveness (online products/tools public access data occupationalproducts/tools, public access data, occupational crosswalks, technical reports, etc.).
Linkages to multiple information sources and databases.
Strengths
Dynamic system allows for updating, capable of incorporating new jobs and descriptors.
Provision of online training supports for use of database.
Use of SOC as occupational classification basis facilitates pmapping to other occupational information/data systems.
Ongoing research program provides knowledge base for ongoing content and methodology enhancements (e gongoing content and methodology enhancements (e.g., improved ways to identify high-growth and new/emerging occupations).
WeaknessesOccupational categories (812) are too broad; data not specificOccupational categories (812) are too broad; data not specific
enough (e.g., compared to 12,000-job DOT) and hence not suitable for selection and other applications.
Descriptors too general; tools/mechanisms needed to permit finerDescriptors too general; tools/mechanisms needed to permit finer level of occupational classification where desired by users.
Some descriptor sets are insufficiently differentiated, too abstract, and lack construct validityand lack construct validity.
Context analysis information may not provide sufficient information for considering job adaptability for dislocated or disabled workersworkers.
Conceptual/definitional overlap of some descriptor elements.
WeaknessesL k f d f d li i dLack of necessary degree of data currency; limited
ability to capture new and emerging occupations.
Establishment sampling method leads to under-sampling of small establishments (while employment in such organizations in growing), and o erlooks self emplo ed part time andand overlooks self-employed, part-time, and contract employees (all growth employment areas).
Sampling of only U.S.-based workers may limit data utility for multinationals.
WeaknessesQuestionable reliability and validity of some descriptor dataQuestionable reliability and validity of some descriptor data
due to use of single-item ratings of broad constructs.
Ratings subject to method effects—spurious correlation of g j presponses (e.g., level and importance) when data are collected via same instrument.
Use of incumbents inflates ratings (i e socially desirableUse of incumbents inflates ratings (i.e., socially desirable responding).
Low inter-rater convergence for some descriptors.g p
No “future-oriented” component to data collection.
WeaknessesL k f bli d l l d (i i ddi iLack of public access to respondent-level data (i.e., in addition
to aggregated occupation-level data) to enable generation of alternative analyses and support research
No public use files or licensing arrangements.
Difficult to match Content Model elements with the items used t th d ith th i t d d t b dto measure them and with the associated database codes
Differential degrees of variability among jobs/titles comprising an O*NET Occupational Unit (OU) may result in differentialan O NET Occupational Unit (OU) may result in differential degrees of accuracy (validity) of OU descriptor ratings when (accurate) SME data from different jobs/positions are combined.
Aggregation IssuesO*NET i l d 1 102 ti ( ll t d tO*NET now includes 1,102 occupations (collects data
on 965)
SOC 2010 includes 840 occupationsp
Since 2006, O*NET has added occupations-- more “breakouts” of SOC occupations and green occupations
Some O*NET users need these disaggregated data and would welcome further disaggregation
Other users need aggregated occupational categories aligned withOther users need aggregated occupational categories aligned with the SOC
The panel did not agree about the appropriate level of aggregation
Aggregation Issues
Recommendation: Assess benefits and costs of changing the occupational classification psystem
Data Collection Issues
Conclusion: The O*NET Center uses a multi method sampling approachmulti-method sampling approach, collecting data from different types of respondents who may or may notrespondents who may or may not represent the work performed in that occupation The impact onoccupation. The impact on measurement error is unclear.
Data Collection Issues
Conclusion: The construct validity of the taxonomies of descriptors variesthe taxonomies of descriptors varies across the content model domains
Improving Database Quality
Conclusion: Over the past decade, DOL has achieved its goal of populating O*NET with updated information, but short-term policy agendas have sometimes reduced focus on core database activitiesfocus on core database activities.
Recommendation: Focus resources on core database activities, leaving development , g pof most new applications and tools to others.
How O*NET Data is Used for Socioeconomic Analysis
Primary research data sets (Census, BLS, NCHS) used for socioeconomic status analysis provide measures of human capital (education, occupation, experience, gender, race, place of birth) but no information on job tasks and skills.
To overcome this limitation researchers imputeTo overcome this limitation, researchers impute task requirements from O*NET to person-level observations from surveys.
How O*NET Data is Used for Socioeconomic Analysis-- The tasks that workers perform on the job are
significant predictors of hourly wages. (Autor and Handel 2009)and Handel, 2009)
-- Job tasks vary substantially between occupations (Autor and Handel 2009)occupations (Autor and Handel, 2009)
-- Variation in job tasks among workers in the same occupation is systematically related tosame occupation is systematically related to race, gender and English-proficiency (Autor and Handel, 2009)
How O*NET Data is Used for Socioeconomic Analysis
Other studies classify O*NET job titles into SES strata based on educationalSES strata based on educational requirements and responsibilities
-- Analysis of work injury risk by SES showed higher risk in lower SES categories (D’Errico et al, 2006)
-- O*NET can be used a job level source of psychosocial exposure information for healthcare typical jobs (Cifuentes 2006)(Cifuentes, 2006)
How O*NET Data is Used for Socioeconomic AnalysisStill other studies use data collected by O*NET for
job classification for analysis of health issues
-- O*NET noise exposure data were assigned to NHANES occupational data (Choi et al, 2011)
-- O*NET has good predictive validity for health outcomes compared with MIDUS data (Meyer et al, 2011)
-- O*NET has been used in 28 studies to estimate work exposures related to health and safety outcomes (Cifuentes et al, 2010)
ReferencesA t D H d H d l M J P tti T k t th T t H C it l J b T k dAutor, D.H. and Handel, M. J., Putting Tasks to the Test: Human Capital, Job Tasks and
Wages, June 2009
Choi, Y., Hu, H., Tak, S., Mukherje, B., and Park, S., Occupational Noise Exposures Assessment Using O*NET and Its Application to as Study of Hearing Loss in the USAssessment Using O*NET and Its Application to as Study of Hearing Loss in the US General Population, Journal of Occupational and Environmental Medicine, July 2011
Cifuentes, M., Socioeconomic Status and Psychosocial Working Conditions in Healthcare Workers Thesis University of Massachusetts Lowell 2006Healthcare Workers, Thesis, University of Massachusetts Lowell, 2006
Cifuentes, M., Boyer, J. Lombardi, D.A. and Punnett, L., Use of O*NET as a Job Exposure Matrix: A Literature Review, American Journal of Industrial Medicine, 2010
Meyer, J., Cifuentes, M, and Warren, N., Association of Self-Rated Physical Health and Incident Hypertension with O*NET Factors: Validation Using a Representative National Survey, Journal of Occupational and Environmental Medicine, February 2011
Tom PlewesSenior Program OfficerSenior Program Officer
Committee on National StatisticsThe National Academies
Room 1138, 500 5th Street, NWWashington, DC 20001
O: 202-334-3454C: 703-408-6285
@[email protected]: http://www.nationalacademies.org/cnstat/