Master thesis: Evaluation of multi-tool and multi-agent systems
Ericsson ABNy
Sammanfattning
This role focuses on evaluating agentic systems within enterprises, particularly in telecommunications, where precision is crucial. The position involves generating a comprehensive set of evaluation questions, creating an automated logging framework, and analyzing system performance through multiple evaluations. The work is based in Stockholm, Sweden, and offers the chance to collaborate with a diverse team at Ericsson, tackling complex challenges in technology.Det här erbjuder vi
Opportunity to work on innovative solutions to complex problems.Collaborative environment with a diverse team.Commitment to diversity and inclusion in the workplace.
Ansök senast: Öppet tillsvidare
Publicerad: 2026-09-18
Beskrivning
Join our Team
About this opportunity
Enterprises are now having many agentic systems with multiple agents interacting with each other and multiple tools being available to the agent for use. These tools are a mix of deterministic systems, vector embedding search, as well as generative models. They may be multimodal covering text, image, graph, code, etc. in both input and output.
The main challenge is evaluation of such systems esp. given the non-deterministic nature and often the large number of tools available. Additionally, for tools which are probabilistic / vector search multiple results are returned some of which are wrong. These problems become pronounced for specific tasks, like in problems in telecommunication systems where preciseness of answer is very important.
What you will do
The skills you bring
Why join Ericsson?At Ericsson, you'll have an outstanding opportunity. The chance to use your skills and imagination to push the boundaries of what's possible. To build solutions never seen before to some of the world's toughest problems. You'll be challenged, but you won't be alone. You'll be joining a team of diverse innovators, all driven to go beyond the status quo to craft what comes next.
What happens once you apply? Click Here to find all you need to know about what our typical hiring process looks like.Encouraging a diverse and inclusive organization is core to our values at Ericsson, that's why we champion it in everything we do. We truly believe that by collaborating with people with different experiences we drive innovation, which is essential for our future growth. We encourage people from all backgrounds to apply and realize their full potential as part of our Ericsson team. Ericsson is proud to be an Equal Opportunity Employer. learn more.
Primary country and city: Sweden (SE) || Stockholm
Req ID: 790724
About this opportunity
Enterprises are now having many agentic systems with multiple agents interacting with each other and multiple tools being available to the agent for use. These tools are a mix of deterministic systems, vector embedding search, as well as generative models. They may be multimodal covering text, image, graph, code, etc. in both input and output.
The main challenge is evaluation of such systems esp. given the non-deterministic nature and often the large number of tools available. Additionally, for tools which are probabilistic / vector search multiple results are returned some of which are wrong. These problems become pronounced for specific tasks, like in problems in telecommunication systems where preciseness of answer is very important.
What you will do
- Generate a master set of questions for agentic evaluation covering a wide range of topics, types and complexity
- Create an automated logging framework so that every tool call's input and output is logged
- Evaluate the system input/output with multiple runs of master set of questions and observe patterns between runs and across questions
- Check on LLM-as-a-judge evaluation and see possibilities and how to calibrate scores
The skills you bring
- Background in EE/Computer Science/Mathematics or related
- Familiarity with wireless communications / signal processing
- Proficiency in Machine Learning and deep learning
- Good programing skills - C++, Python
- Familiarity with programming environment in Linux
Why join Ericsson?At Ericsson, you'll have an outstanding opportunity. The chance to use your skills and imagination to push the boundaries of what's possible. To build solutions never seen before to some of the world's toughest problems. You'll be challenged, but you won't be alone. You'll be joining a team of diverse innovators, all driven to go beyond the status quo to craft what comes next.
What happens once you apply? Click Here to find all you need to know about what our typical hiring process looks like.Encouraging a diverse and inclusive organization is core to our values at Ericsson, that's why we champion it in everything we do. We truly believe that by collaborating with people with different experiences we drive innovation, which is essential for our future growth. We encourage people from all backgrounds to apply and realize their full potential as part of our Ericsson team. Ericsson is proud to be an Equal Opportunity Employer. learn more.
Primary country and city: Sweden (SE) || Stockholm
Req ID: 790724
Ansök till tjänsten
Master thesis: Evaluation of multi-tool and multi-agent systems
Ny
OM FÖRETAGET
Ericsson AB
