AI in the office of finance: how Intito built agents that meet the finance standard
At Intito, we have spent the past year solving what turns out to be one of the harder problems in applied AI: getting a financial figure exactly right, every time. This article shares what we have learned — and what a panel of finance practitioners told us about what it will take to actually trust AI in the office of finance.
For the past few years, AI has been a regular theme on the Finance & Data Summit stage. Two years ago the topic was generative AI. Last year it was agents. This year, Intito's AI Lead Janus Timperi traced the whole arc, from the first tentative experiments to where the technology actually stands now, and a panel of finance and technology practitioners picked up the same thread: AI and agentic AI are no longer coming to the office of finance. They are here. The harder question is what it takes to trust them.
Key takeaways
- Finance has a higher bar than most domains. In finance, an answer is either right or wrong — never 'roughly right'. AI agents must meet this standard before they can be trusted in production
- The finance role is not disappearing, it is shifting. Controllers will orchestrate automation rather than do the underlying tasks; the CFO's role strengthens as finance processes become more real-time.
- Excel is hard to kill, but the direction is clear. Even the most capable AI agents conclude that Excel-dependency is a risk worth reducing, even if it remains a staple in most organisations.
- Cost is falling fast. A token costs approximately a tenth of what it did in 2023. The business case for AI in finance will keep improving.
- Demos are easy; production is hard. Open models score 50–60% accuracy on finance queries. After rigorous development, Intito's agent reached a level where harder test questions had to be invented to make it fail. The gap between a convincing demo and a trustworthy production tool is where the real work happens.
Why finance is a difficult place for AI
Timperi opened with the reasons finance is genuinely hard for AI, and they set up everything that followed.
First, the data is sensitive. You do not paste your own figures into just any tool and see what happens; ChatGPT in a browser is not the right home for them. Second, the numbers have to be exactly right, down to the decimals. Generative AI is probabilistic by nature. It generates, and it can hallucinate. A few correct figures, followed by a few invented ones, do not just fail; it actively points you in the wrong direction. And third, with agents, a lot can happen between the question and the answer a person finally sees, so it matters enormously to know where the numbers came from and whether they are correct.
That single standard, that in finance an answer is either right or wrong and never "roughly right," runs through the entire story.
Three years in fast-forward
Timperi’s presentation traced the arc almost like a time-lapse of the industry’s evolution. Three years ago, with the first public versions of ChatGPT, you went to a chat window on a website. It did not integrate with anything, it could not fetch your data, and for finance the verdict was simple: there was no real use for it yet, just an open question of whether there might be.
Two years ago, came retrieval-augmented generation, giving a language model your data to work from rather than relying on what it had memorized. Intito built early experiments, an ESG-reporting agent that could pull answers from regulation, and first trials of whether generative AI could handle the harder, multidimensional data in Planning Analytics. The demos looked convincing, but they were effectively hard-coded. You had to ask exactly the right question in exactly the right way, and anyone else touching it got nothing sensible. The error rate was high, and the interfaces had to be built by hand.
A year ago, agents arrived in practice. An agent differs from a plain model in that it is given a list of tools it can use; the language model is the brain, with orchestrating code around it, and it reasons independently across several steps before producing an answer. Frameworks meant one no longer had to code everything from scratch, and the newer models were finally capable enough to reason which tool to use when.
Then came last year, and the pace surprised even the people building it. New frontier models, the ability to put a capable agent directly into Planning Analytics, and properly formatted output instead of the clumsy text boxes of before. As Timperi put it, to a user the screen does not look dramatically different, maybe a bit nicer, but under the hood the difference is enormous.
The real work is measurement, not demos
The central lesson of Timperi's talk is easy to miss because it sounds mundane: a demo is easy to build with agents; getting it to answer finance figures exactly right, without hallucinating, is the hard part. Intito has spent the past year largely on that problem.
The team began with IBM technology, partly so the data stayed in known data centres rather than a public model, and partly because smaller models are cheaper and faster even if less capable. When a couple of customers tried the agent on real production data, the limits showed: the smaller models' reasoning ran out, and so did the reliability of their answers. The question became whether it was an architecture problem, a model problem, or simply whether the technology was good enough at all.
The answer was rigor. Intito built what amounts to a workbench for tracking how agents perform and analyzing why they fail, with agents built on top to read the step-by-step logs and surface where things go wrong. The development loop tightened: Timperi describes a coding assistant making changes, a test data set and test questions run several times, one agent analyzing where the results break, another checking whether the code can be fixed, and a human in the middle deciding whether there's anything to add. Four iteration cycles in a single day became possible, and reliability climbed.
The numbers tell the story. On a correctness measure, open models such as Llama 4 Maverick and a GPT open model scored around 50 to 60 percent, fine in many settings, but as Timperi noted, a fifty-fifty chance of a correct answer does not warm the heart in the world of finance. An early Claude Sonnet version scored roughly 85 percent. After building the agent lab and iterating, on the same model but with better instructions and better code, accuracy rose to the point where the team now has to invent harder questions just to make the tests challenging enough to fail.
What an agent can actually do today
In the live demo, the agent lived in Watsonx Orchestrate (IBM’s AI orchestration platform), reachable from Teams, Planning Analytics and other channels, all talking to the same agent with Planning Analytics as its data source. It could list its own capabilities, answer in Finnish, English or German, retrieve operating revenue by company for a given month, and run outlier detection across a two-year time series to flag anomalies worth a closer look in monthly reporting.
The demo also showed why bigger models feel slower. Asked whether there are anomalies, the model has to work out which data is meant, run the algorithms, then analyze the results, and each of those small steps queries the model again. Bigger models are smarter but slower, so a query might take a minute or two. And the conversation can continue naturally: pick one company, ask for the same figures broken down by account or by month, and the agent understands the generic follow-up and responds.
One point Timperi stressed has architectural weight. You do not always need to load data into a warehouse first. Intito's own HR and marketing agents reach into HubSpot and SharePoint through interfaces and fetch what they need when they need it, and Planning Analytics is, to the agent, simply a modelled data set available through an interface.
Frequently asked questions
Why is finance a particularly difficult domain for AI?
The office of finance relies on exact figures. Generative AI is probabilistic by nature and can hallucinate — producing a few correct numbers followed by invented ones. This is why getting AI right in finance requires rigorous testing and architecture, not just capable models.
What is retrieval-augmented generation (RAG)?
Retrieval-augmented generation (RAG) gives a language model access to your own data — documents, reports, databases — rather than relying only on what it learned during training. For finance, this is critical: it means the model can answer questions about your actual figures, not generic information.
What is the difference between AI and agentic AI?
Agentic AI differs from a plain language model in that it is given tools it can use autonomously — querying databases, running calculations, fetching data — and reasons across several steps before producing an answer. Unlike a chatbot that responds when asked, an agent can handle a process end to end.
The technical picture is only part of the story. The panel discussion that followed — with finance leaders from IBM, DNA, and Varma — tested these ideas against lived experience. Read part two: What finance leaders say about AI.
Interested in what agentic AI could mean for your finance team? Intito works with organisations across the Nordics to move from AI experiments to production-grade tools. Get in touch to see what is possible.
By Janus Timperi, AI Lead, Intito