Reflecting on the role of data in the ETF's work: Interview with Manuela Prina
The use of data in the ETF’s work has increased exponentially in recent years, with the evidence generated a core part of monitoring and assessment, and for informing decision-makers. The rise in data use has come at the same time as the roll-out of Big Data and, more recently, generative artificial intelligence.
To reflect on the use of data and new technologies, the ETF held internal Data Days for its experts to get a better grasp of what is possible, and what the future of data use might be. Manuela Prina, Head of Skills Identification and Development Unit at the ETF, gives us an insight on what was discussed at the Data Days and the importance of data.
A fixation on data
The importance of data may seem evident to many, particularly when it comes to the development of evidence-based decision-making and how that impacts stakeholders, beneficiaries and the system at large. But to some, said Prina, the ETF appears “obsessed with data.”
"While experience and creativity are important components of our work, they are not enough in themselves. Our work has to be grounded in data, because it substantiates the advice we give and offers options for decision-making. This is a core value of our professionalism and credibility,” she said.
Data Days
The internal gathering reflected on what the ETF does, how data is managed, and the approach taken. It was also an opportunity to look at innovations and define certain priorities.
“The idea was to be more familiar with the enormous quantity of information and data, and to talk about data in a larger sense. We are also in a transition period in terms of documentation management systems and databases. There was a need to take stock of the variety of data we have available, and to be more effective and efficient in making use of data in different services, without having to reinvent the wheel every time,” said Prina.
Data Days had three objectives. The first was to reflect on data at large, while the second objective was a deep dive into certain data work to enable ETF experts to better understand how such data can be exploited.
“This comes more naturally to some experts, who are more creative about data and its use for different objectives, from training to advice, to comparing experience, while for some others this process is more laborious. We wanted to share how some experts exploit and squeeze data to have the best outcomes, and to foster peer learning,” she said.
Generative AI
The third objective concerned generative artificial intelligence (AI), which can produce various types of content, including text and synthetic data in response to prompts. The public deployment of large language models (LLMs) like AI chatbot ChatGPT, over the past year has caused much excitement as well as consternation around what the technology can be used for.
“We pictured the future, in particular how we treat in-house knowledge and data creation, the management and use of data, and had a brainstorming on AI,” said Prina.
To enable participants to see what ChatGPT is capable of, a digital engineer programmed the tool to work with ETF products and data to generate output.
Participants reflected on how AI could speed up certain processes. For instance, in the future generative AI could lead to a shift in the ETF’s role as a producer of data to becoming a prompter and verifier. There is also the possibility of providing greater access to ETF data to external partners, who could then prompt their own questions.
AI and its future uses
Integrating AI into the ETF’s processes would however require the assessment of opportunities and the risks to be managed. “It is clear from the work we did during the Data Days that we are not there yet, in terms of using AI to create databases or to use AI to accelerate certain analytical work, but that doesn’t mean it will not happen in the future,” said Prina.
Regarding analytical work, AI could be used to assess certain areas, such as what skills are required in a certain sector.
“We could take 100 reports on say the energy sector in a certain country or area, and prompt AI to find answers from the reports. This might generate the acceleration of an output to six or eight months instead of two years. That could really have value when requests from stakeholders at a policy level are multiple, and we need to accelerate our work,” she said.
Big Data
The uses of Big Data was a further area of reflection. Prina said that before the Covid pandemic, the ETF had started to assess online job offers in several countries – including Morocco, Tunisia, Egypt, Georgia and Ukraine – to reveal dynamics in the levels of skills required and educational levels needed in the labour market. “Until recently this looked almost futuristic,” she said.
In Ukraine for instance, online job vacancies “gives us quite a good overview about how the labour market is moving. You can see the moments of increased bombings or attacks, when there is a drop in the labour market, and when there are moments of calm, when there is an up-tick. It is quite revealing,” she said.
Prina noted that while such data needs to be considered in relation to the informal job market, big data can be useful in analysis.
“Going forward, this could be an integral part of how we monitor a labour market.
“How the ETF evolves in tandem with the data evolution is an exciting and challenging topic that we are fully embracing,” concluded Prina.