Big data analyses of the labourmarket

Emilio Colombo

Università Cattolica del Sacro Cuore

CRISP

ESCO Conference, 09/10/20171

What lays ahead?

• The world economy has been invested by a technologicalrevolution which is reshaping the global economic system

• The primary factor in change is the shifting “from aneconomy centred on producing physical goods to onecentred on innovation and knowledge (...) For the first timein history, the factor that is scarce is not physical capital butcreativity.“ (The new geography of jobs, E. Moretti, 2012)

2

The challenges of the future

• Technology, population ageing, migration flows are affectingthe labour market with a strength and a speed which areunprecedented

• Frey and Osborne (2013) 47% of Jobs will disappear in thenext 25 years.

• World Economic Forum (2016): 65% of children enteringprimary school today will ultimately end up working incompletely new job types that don’t yet exist.

3

The challenges of the future

•What skills will be required by the labour market in the future?

•Will the labour force be up to the standards of the demand side?

•What sort of mismatches will arise and where?

•Skill mismatch has a lot to do with the difficulty in correctly define and classify skills (mainly soft skills).

4

LABOUR MARKET

PEOPLE

What are the most required skills

for my job? Where is the demand for my occupation (company,

sector, territory)?

P.A.

Which

training/educational

policies are needed?

COMPANIES

What are the evolution

trends in skills for the

occupations of interest and how can I improve the

training of my employees?

EMPLOYMENT

AGENCIES

Which profiles the

companies are

looking for in my area / sector...?

5

Work mobility

The change, with greater speed, of skills required within the same occupation

The polarization of occupations

The emerging of new occupations Global markets vs local markets

6

Features of today’s labour market

New policies, new tools

• We need new policies at local, national and internationallevel

• But before we need to assess and know better the size andmagnitude of these phenomena and their implications forthe labour market

• This calls for new data and new tools.

7

New tools

• We need forecasting tools to identify the most relevanttrends. This is what Cedefop does since 2007• But forecasting tools are necessarily imprecise about

the features and skill requirements of the jobs of thefuture

• We need tools to investigate firms’ skill needs. This is whatCedefop has done with the European Skills and Jobs survey.• However surveys are rigid and lengthy tools

• We need tools able to detect the change in the labourmarket as it occurs. We need real time tools. 8

Problems with skill surveys

1. They are costly, considering direct (implementation) and indirect (opportunity cost) costs.

2. Their implementation is not easy, thus they cannot have a high frequency.

3. They have a top-down approach, i.e. soft skills and occupation-specific skills are generally pre-defined.

4. What you have is what you designed. Not more sometimes less.

9

How web vacancies may help

1. Less costly to be implemented. High initial cost but low marginal cost.

2. No implementation lag. Almost real time data.

3. Bottom-up approach, richer classification especially useful for some soft skills and particularly for occupation-specific skills.

4. Information is always there. Unless there are storage problems you can also go back and retrieve what you missed.

10

Big Data: from data to knowledge

The value added of big data is the ability to answer the questions of different stakeholders with more and timely data!

http://www.wollybi.com/

11

How does it work?

Collect Web Job VacanciesScrape and collect data from several sources, previously ranked to guarantee high quality (egpresence of publication/update timestamp, territorial granularity, etc)

Data transformation and cleansing

Fills the data into our lossless data model performing data cleaning activities (eg, duplicate detection)

ClassificationUse supervised machine learning approaches to classify Web Job Vacancies according to ISCO/ESCO classification system

Skills extractionUse top AI algorithms (eg, semantic similarity measures, IE techniques, NLP) to extract, recognise and link skills to occupation profiles

Data VisualisationKnowledge representation of data evolution over time by means of data visualisation tools, having in mind the stakeholder features

12

Some useful insights

• Insights: big data let the data speak and allow us to grasp what ishappening now, and understand how these changes are manifestingthemselves.

• In the next slides some insights will be presented to give an idea ofthe informative potential:• Insight 1: how widespread are ICT skills in occupations?• Insight 2: what are the new emerging occupations?• Insight 3: how the demand for skills has changed over recent years?

• These question are difficult to answer particularly with a top downapproach

• So let’s ask the data

13

The Skill Digital Rate (SDR)

measures the incidence of

digital skills in a given

occupation.

For ICT occupations, the

average SDR is 68%, higher

values are recorded in new

occupations.

Fonte WollyBI – Italian Labour Market Digital Monitor

The Skill Digital Rate in ICT occupations

Skill Digital Rate percentage 14

Skill Digital Rate pct

For non-ICT occupations, SDRs is

growing for administrative,

management activities and

market development activities

(e.g. human resources,

accounting and marketing).

Source WollyBI – Italian Labour Market Digital Monitor

The Skill Digital Rate in non-ICT occupations

15

Freight handlers

Human resource managers

Accounting associate professionals

Management and organisation analysts

Advertising and marketing professionals

Digital skills in non-ICT occupations

Prevalence of Digital Basic Skills and of computer

intermediation

LOW SKILL DIGITAL RATE

Uso PC, Word Proc. , Social Network, Wordpress

Prevalence of digital skills Applications and

Techniques

HIGH SKILL DIGITAL RATE

Database

ERP CAD Java HTML5 CMS

PHP

Uso PC, Word Proc. , Social Network, Wordpress

The composition of digital skills

in non-ICT professions changes

according to the Skill Digital Rate:

the lower the SDR occupations

the more basic the digital skills

(i.e. Basic and Information

Intermediation). The higher the

SDR, the more technical the skills

(i.e. application and techniques).

Source WollyBI – Italian Labour Market Digital Monitor 16

The «new» emerging occupations

Data Scientist Cloud Computing Cyber Security Expert

Business Intelligence

Analyst Big Data Analyst Social Media Marketing

Trend changes:February-April 2017 compared to the

year 2013: + 278%

More than 7 thousands vacancies

from February 2013 to April 2017

172013 2014 2015 2016 2017

Skills - Data Scientist Skills - Cloud Computing

Relational Abilities Team Working

Java Machine Learning Reporting

Data Visualization Organizational Abilities

DataWarehouse DataBase

Statistical Models Phyton

Professionalism Sql Knowledge ERP Applications

Data Mining Data Analysis Office Package

SAS Programming Management Knowledge

Business Intelligence & Analytics Pc Use

Sql Pc Use Linux Telecommunication Knowledge

Team Working Management

Knowledge Professionalism

Java Html

Relational Abilities Vmwar DataBase

Windows Server Scripting

Skills - Cyber Security ExpertSkills - Business Intelligence

Analyst

Relational Abilities Team Working

Responsibility Sense Security Certification

Professionalism Linux Organizational Abilities

Telecommunication Knowledge

Unix Security Systems Java

Firewall Problem Solving

Malware Analysis Pc Use

Team Working SAS Programming

Office Package SqlProfessionalism Data Modeling

Java Dbms

Relational Abilities DataBaseData Analysis ETL

Business Object QlikView

Business Intelligence

Reporting

Skills - Big Data Analyst Skills - Social Media Marketing

Reporting SAS & R Jboss

Relational Abilities DataBase Data

Analysis NoSQL Java DataWarehouseBusiness Intelligence SQL Server

Team Working Cloudera ClouderaHadoop Professionalism

Python Sql

Team Working Indesign

Photoshop SEOOffice Package Professionalism

Relational Abilities Web Edit

Graphic Programs Html5Management/OrganizationTeam Working Marketing Knowledge

Google Adwords

Social Network Wordpress

Changes skill demand 2013 vs. 2017

Soft skills are growing in importance due to the relevance of collaboration and relational abilities even in highly ICT-specialized jobs.

The occupation description by ESCO: Database designers and administrators design, develop, control, maintain and support the optimal performance and

security of databases.21

2521 - Database Designers and Administrators

• DATA WHAREHOUSE: -11%.

• ORACLE: -7 %

• LINUX: -4 %

• MANAGEMENT & ORGANIZATION:

+21%.

• MANAGE THE RELATIONS WITH

CLIENTS: +18 %

• ATTITUDES & VALUES: +12 %

Changes skill demand 2013 vs. 2017

22

2431 – Advertising and marketing professionals

• ADOBE PHOTOSHOP: +3 %

• GRAPHIC EDITING SOFTWARES: +3 %

• ATTITUDES & VALUES: -7 %

• MS OFFICE: -3 %

• THINK CREATIVELY: -2 %

Conversely, the need for hard skills is growing as they convert traditional jobs towards digital.

The occupation description by ESCO: Advertising and marketing professionals develop and coordinate advertising strategies and campaigns, determine the

market for new goods and services, and identify and develop market opportunities for new and existing goods and services.

Big data allows big KNOWLEDGE

Pros of Web Data:

1. Data-driven approach that allows:

• The analysis of occupations and skills dynamics

• The identification of new trends and new emerging occupations

2. No implementation lag. Near real-time analyses.

3. Less costly to be implemented vs surveys. High initial cost but low marginal cost (maintenance and evolution).

The main featureof a Big Data approach is the real-time collectionof all relevant data and “let the data speak";

23