Thesis: Cost-Aware Tiered Website Interaction with Calibrated Decision Models

High Level Description
Web-based application utilization by AI agents, operating on behalf of users, has observed dramatic growth in the past several years globally. Various global internet utilization benchmarks, such as HUMAN Security and Cloudflare Radar, indicate that in the last few years, agentic AI HTTP requests are beginning to outnumber those made by human website users. This newly emerging web application utilization paradigm has significant consequences for both the agent operators and the infrastructure maintenance of websites, as AI agent utilization patterns and volumes differ greatly from those of regular users. This master's thesis aims to explore how AI-powered website assistants can reduce token usage, cost and response time while maintaining task success. In collaboration with byBrick Interface, you will develop an assistant that handles straightforward requests using a lightweight decision model and passes more demanding requests to progressively larger language models (LLMs).

Project Description
The project will involve designing, implementing and evaluating a tiered website assistant for tasks such as finding products, filtering search results, locating orders or submitting reviews. The assistant will select structured website actions where possible and involve larger models when confidence is insufficient or generated language is needed.

The research will focus on how reliably model confidence can guide these escalation decisions. You will compare the proposed approach with LLM-only solutions, measuring task success, token usage, cost and latency, including the overhead of routing between models.

The work will include a literature review, prototype development and experiments using representative website requests. byBrick Interface will contribute a relevant use case and regular feedback throughout the project.


Who are we looking for?

• Students pursuing a master's degree in Computer Science, Software Engineering, Artificial Intelligence, Machine Learning or a related field, preferably working as a pair.
  
• Good programming skills, for example in Python or JavaScript/TypeScript, and an interest in web applications and APIs.
  
• Foundational knowledge of machine learning and an interest in LLMs, AI agents and experimental evaluation.
  
• The ability to work independently, critically analyze results and communicate findings clearly. Experience with Docker or model APIs is beneficial.
  
• Expected graduation in June 2027.
  
• Fluency in English, both written and spoken.
  
• Fluency in Swedish, both written and spoken, is normally required. Candidates who do not meet this requirement are still encouraged to apply, as language requirements may be assessed individually.
  
  

Purpose
The purpose of this research is to determine when lightweight decision models can handle website interactions reliably and when larger LLMs are needed. The expected outcomes are a working prototype, a reproducible evaluation suite and practical guidance for developing website assistants that balance operating cost, response time and task success.

An Exciting Journey with Knightec Group
Semcon and Knightec have joined forces as Knightec Group. Together, we are Northern Europe's leading strategic partner in product and digital service development. With a unique combination of cross-functional expertise and a holistic business understanding, we help our clients realize their strategies - from idea to complete solution.

Practical Information
This is a Thesis student position, located at our office in Västerås. Start date is the beginning of January 2027.

Please submit your application as soon as possible, but no later than 2026-11-30. If you have any questions, you are welcome to contact Aldin Berisa and Edin Jelacic. Note that due to GDPR, we only accept applications through our careers page.