SAP GUI AI Agent: Architecture & Technical Details
Artificial Intelligence Agents are autonomous programs designed to perform tasks intelligently without human intervention. Leveraging advancements in large language models, these agents can understand and generate human-like text, enabling them to tackle complex challenges across various domains. I think it’s worth saying that most agents are also able to use the tools provided to them by the developer. As a result, agents often have specializations that frequently depend on the tools they’re able to use.
For example, there are agents that can do a research for you, manage your web browser, set up appointments, or even trade cryptocurrencies for you. The possibilities are essentially limitless, and everything depends on the intelligence of the large language model and the tools with which the agent has been equipped.
To clarify, let’s compare agents to popular chat systems. In a typical chat, when a user asks a question, the AI responds, and then it’s back to the user for the next input. This back-and-forth creates a conversation. An agent, on the other hand, usually differs as it can execute multiple tasks on its own before returning to the user. This means it can accomplish more and utilize various tools before getting back to the user.
This is just one way to understand the broad concept of an “AI agent.” However, for this post, we’ll go with this definition.
SAP GUI AI Agent
SAP GUI AI Agent is a program designed to interact with the graphical user interface of the SAP system. It can handle a wide variety of tasks, even those it hasn’t been explicitly programmed to perform in advance. This is made possible by the vast knowledge that large language models have.
The agent was created in November 2024, and its functionality was showcased on the YouTube channel Subdalf. The inspiration for its creation came from the occasional solutions capable of using a web browser. I thought that if it’s possible to use artificial intelligence to navigate the interface for browsing web pages, then it should also be possible to operate in SAP GUI.
The agent was developed in November 2024 and its capabilities were showcased on the Sapdalf’s YouTube channel. The idea came from innovative projects that use a web browser. I figured if AI can navigate web interfaces, then it should be able to work with SAP GUI too.
In the original version, the agent was given a rather limited number of tools, yet it operated surprisingly efficiently. This aroused the interest of many people in the SAP community and encouraged me to further develop the program. I was genuinely surprised by how efficiently the agent behaved once it had the right tools. I hadn’t anticipated such performance, as I mentioned in my previous blog post.
In December 2024, I released a video showcasing an upgraded version of the agent. I added new tools that expanded its capabilities, allowing it to create programs in ABAP. This version was also more cost-effective because I worked on reducing the need for costly screenshot analyses in favor of text analysis. I also promised to dive back into the technical details of this project. So here we are with today’s post.
Agent Architecture
AI
To truly appreciate the capabilities of the SAP AI agent, it’s essential to understand the key components and frameworks that shape its architecture. At its core, the agent is built using Python, a widely recognized and versatile programming language in the AI domain. Python’s extensive ecosystem and supportive libraries make it a natural choice for developing AI tools like this one.
If Python is the undisputed leader in AI solutions, then LangChain is the go-to library for building LLM-based projects. LangChain simplifies the process of integrating AI models, particularly for conversational and language-heavy tasks. Complementing this is LangGraph, an extension tailored for creating agents that offers developers a balance between simplicity and precise control. There are currently many frameworks, such as Smolagents, that offer greater simplicity but less control. Here, control over operations is very important, which is why it is worth using a well-established solution.
What are libraries without a large language model, though? There’s a wide array of choices out there, but I tend to stick with GPT-4o, as it’s reasonably priced and offers impressive capabilities. I’m also thinking about checking out other models like o3 Mini. With the LangChain library, switching models, no matter the provider, should be a breeze.
