Big Data training for LMI - Online Job vacancy analysis virtual training programme - 8, 10 and 15 June

Big Data for Labour Market Intelligence

Training programme: 8, 10 and 15 June 2021 - Virtual 

ETF project Big Data for Labour Market Intelligence (LMI)

Started in the second half 2018 the ETF innovation project elaborated first a brief methodological handbook “Big Data for labour market intelligence: an introductory guide” (2019). A training programme for experts of the “Make it Match Network” of Eastern Partnership was a success in November 2019. In 2020 the full data system for analysis of OJV was established, with data dashboards available: Tunisia and Ukraine. The dashboards are updated with data from online job vacancies collected until 31 Mach 2021. The data system is focused on demand (online job vacancies – OJV). 

The ETF Big Data for LMI project continues in 2021, with addition of a new country and the continuation of Online Job Vacancies (OJV) data ingestion, classification and analysis for Tunisia and Ukraine. Many new questions and queries can be analysed on the basis of this growing database of OJVs, as predicted by the paradigm “let the data speak”. New insights will be identified from the vast data set, and “Green Dashboard” developed.

The Training Programme

The three-day journey of this virtual training programme is conceived to respond to these questions:

  • But what is Big Data analytics for Labour Market Information Systems?
  • How to explore and create value from large volumes of Online Job Vacancies for real-time LMIS?
  • Which dimensions and issues related to skills, occupations, and labour market dynamics questions can be analysed with help of Big Data / Real-time LMI?
  • Which are the requirements to harness these novel data sources and systems by ETF Partner countries?
  • How can Real-time LMI based on Big Data and AI-aided classification be used and complement established national statistics?

Participants: representatives from research and analytical departments of ministries (education, training and labour) and from specialized research centres from all ETF partner countries. Participation in all activities planned in the three-day programme is recommended (mandatory) for all registered participants.

The three-day journey of this virtual training programme mobilises participants 4,5 hours a day and intercalates days for review and preparation of the thematic blocks of the training programme. Please download the agenda (EN-FR-RU) for your easier reference.

  • Day 1: 8 June - Real-time Labour Market Intelligence (LMI): Overview, goals, advantages and use cases.
  • Day 2: 10 June - Real-time LMI: Focus on data Ingestion, processing and classification
  • Day 3: 15 June - Skills Intelligence: use cases, hands-on exercises and lessons learned

Practical application started in 2019 with a feasibility analysis of the web labour markets of Morocco and Tunisia, resulting in a comprehensive report assessing and ranking online job vacancy (OJV) portals. The establishment of an integrated system for data collection, processing, classification, analysis and visualisation was the core of the work in 2020, in two pilot countries (Ukraine and Tunisia).

Working with the data science team of Burning Glass Europe (Italy), the European Training Foundation (ETF) has completed in December 2020 a deciding phase of its innovation project “Big Data for Labour Market Intelligence”. Hundreds of thousands of online job vacancies collected over 8 consecutive months in 2020 (April-December), processed and automatically classified against such international classifications / taxonomies as ISCED 2011, NACE, NUTS / ISO and ESCO provide unique insights on skills and occupational features and dynamics of the Tunisian and Ukrainian labour markets. We say “unique insights” – because of their granularity and real time nature. Some of the many possible angles of analysis are visualized in two countries’ dashboards.

The data system uses ESCO as the reference for machine-classification of skills identified in the hundreds of thousands of OJVs. For the case of Tunisian OJV data we used ESCO – in French and English versions. But for the case of the Ukraine, an additional step was indispensable: translation of ESCO skills into Russian and Ukrainian languages (over 4,000 terms).

Ukraine professional dashboard: visit a multi-dimensional view of all variables in one snapshot.

The particular advantage of OJVs as sources for LMI lies with the fact that they express / represent the employers’ determination of the profiles they need for the purposes of the business or activity in a given period. The machine processing and classification of employers’ own terms and descriptions of skills shows cases of OJVs skills without a direct ESCO correspondence. No surprise: technological and digital transformation of work and skills is much faster than the pace of alignment and update of ESCO.  What to do then? The data science team involved in the project applied machine-learning techniques (e.g., Word2Vec) to enhance ESCO skills, creating a correspondence between a new ‘non-ESCO’ skill with a close (approximate) ESCO-skill. This process and the machine-proposed correspondence is discussed and validated by (human) professionals in the given sector, occupation and technology. Can this technique and approach have a wider application in the context of ESCO updates?

Key outcomes of ETF Big Data for LMI activities, including data dashboards, training programmes, methodological handbook and analyses, have been published at:

A new world of data analytics…

Skills intelligence as business as usual is not enough to understand the direction and extent of the transformation of tasks, jobs, skills and qualifications prompted by a wave of drivers of change, which boosted the digitalisation of most processes in our societies. New data analytics have emerged to advance skills intelligence and complement conventional statistics, surveys and administrative data.

Data is being called the new oil. Digitalisation of processes, services, businesses, personal and social interactions generates a growing mass of data across the globe. Creating knowledge out of large volumes of data, available with high velocity and variety is the major goal of Big Data analysis.

…can be applied for labour market information

Artificial intelligence (AI) and machine learning are not only changing the labour market, but also giving us new tools for analysing the workforce. Job vacancies or job advertisements are published, refreshed, updated in large numbers through websites of different types, size and coverage. Exploring the inherent information of a such large data source has become an objective of research centres and public bodies in a number of countries. These vast data sources are essential to understand the dynamics and functioning of Web Labour Markets, and of changing employers’ recruitment choices.

Big Data analytics can be used to map skills by occupations, to identify obsolete skills, to do predictive analysis of demand for new occupations and skills, and to better capture skills interactions - based on granularity of data and quasi in real time.

In the European Union, since 2016 Cedefop is leading a breakthrough project in this area and created a vast data system based on the analysis and classification of millions of online job vacancies (OJV) of European Union (EU) Member States. In the platform OVATE the results are presented in interactive dashboards of combined variables, and different geographic coverage. In 2021 Cedefop renewed and upgraded the OVATE dashboards and will continue analysis of occupation-skills-sector relationships. On the other side Eurostat is working with Cedefop and takes over the data infrastructure part, to focus on statistics and detailed time trends and focus on territorial location – contributing to the Smart Statistics project of Eurostat. This new phase of the project is jointly steered by Cedefop and Eurostat.

11/04/2021. Contact: Eduarda Castel-Branco -



More information

Tunisia Big Data analysis
Ukraine Big Data analysis
Case study day 1 Eurostat
Case study day 2 Burning Glass career paths
Guide Big Data
Bio Eduarda Castel Branco
Bio Alessandro Vaccarino
Bio Mauro Pelucchi
Presentation Eduarda Castel Branco day 1
Presentation Dimitrios Pikios Day 3
Mentimeter results day 1
Presentation Day 1 training
Presentation day 3 training