When AI Gets It Wrong: What Finance Teams Can Learn from AI’s Biggest Missteps
By Accounting & Finance

Pick any finance publication, conference agenda or LinkedIn feed, and you’ll get the same story about AI. It will speed up the close, sharpen the forecast, free your team from the grind and transform the function. Some of that is true. I’m not here to argue AI has no place in finance, because it clearly does.

What I rarely see is anyone writing about the other side of it: what happens when it goes wrong. Every article I read leans on the benefits. Hardly any of them stop to look at the mistakes other businesses have already made, and those mistakes are the cheapest lessons going. Somebody else has already footed the bill. You just have to read them.

So that’s what I want to do here. I’ve picked three real examples, from a Big Four firm, a fintech and the wider research on how AI projects actually perform. Taken together, they point to the same thing, and it isn’t a technology problem. It’s a judgement problem, and finance leaders are better placed than most to fix it.

The story we’re being told, and what’s missing from it

The adoption numbers look impressive. Gartner’s recent surveys of finance leaders show AI moving quickly from experiment into production across the function.1 But the same research is clear that value isn’t keeping pace. In a March 2026 survey of 204 finance leaders, 63% said their AI implementation had been slower than expected in 2025, and Gartner’s message to CFOs was blunt: stop mistaking deployment for value creation.

That gap between “we’ve switched it on” and “it’s actually working” is where the interesting stories sit.

Mistake one: trusting the output because it looks right

In 2025, Deloitte Australia agreed to partially refund the Australian government for a report it had produced, valued at around A$440,000.2 The report, a review of an automated welfare compliance system, turned out to contain references to academic research that didn’t exist and a fabricated reference to a federal court case. A revised version later disclosed that a generative AI tool had been used in producing it. Deloitte maintained that the corrections did not affect the report’s substantive findings.

What I take from this isn’t that a Big Four firm is careless. It’s how plausible the errors were. A sentence that sounds authoritative, a citation that looks properly formatted, a number in the right range. AI is very good at producing things that look right, and that’s precisely what makes it dangerous in finance, where “looks right” has never been the standard. In our world, the question is whether it ties back to the ledger, the source document, the contract.

The lesson for a finance team is simple and a bit unglamorous. If AI produces something that goes to a board, an auditor, a lender or a client, a named person needs to own checking it, and they need the skills and the time to actually do it. Review that exists on paper but not in practice is how this happens.

Mistake two: treating AI as a substitute for people

Klarna is the example I keep coming back to, because it was so public. The fintech launched an AI assistant in early 2024 and said it was doing the work of around 700 customer service agents, handling about three-quarters of customer chats within a month.3 It slowed hiring and let headcount shrink through attrition.3

Then, in 2025, its CEO told Bloomberg the company was recruiting human agents again, because the cost-led approach had resulted in “lower quality”. To be fair to Klarna, it describes the move as a dual-track approach combining AI with human support, not a step back from automation. But the direction of travel is the point: the cheaper option wasn’t the better one for customers.

I hear a version of this on the finance side of the market too. We spoke to a CFO who said that when someone leaves his team, the first thing he does is check whether anyone else can absorb the work or whether the process can be automated, rather than hiring a replacement. It’s an understandable instinct. But it often goes wrong in two ways. The role may be too niche or too specific for any system to take over. And the team is usually stretched already, so handing them more responsibility burns them out further and leads to more people leaving, which is the opposite of what the CFO was trying to achieve.

Automating a process is a perfectly good decision. Automating a person out of a seat before you’ve understood what they actually did is a different thing, and that’s where I’d urge caution.

Mistake three: switching it on before the data is ready

This is the one that comes up most when I think about finance specifically. A widely quoted MIT study from July 2025, based on interviews, an employee survey and an analysis of public AI deployments, found that around 95% of generative AI pilots in the companies it examined delivered no measurable impact on the P&L.4 It’s worth being careful with that number, because it covers generative AI pilots specifically rather than every kind of AI, and the researchers pointed to how organisations were integrating the tools rather than the quality of the models themselves. But it fits a pattern finance leaders will recognise.

Gartner’s 2024 survey of 121 finance leaders put inadequate data quality or availability, and low data literacy, as the top two challenges to adopting AI.5 And CFO Recruit’s 2026 survey of more than 500 North American CFOs points the same way: 90% said AI was on their agenda, but 70% had no defined plan to put it into practice, and the firm says many finance functions can’t get clean, consistent data out of the systems they already use, so adding AI on top only makes the existing problem run faster.6

If your ledgers, mappings and master data aren’t clean, a more sophisticated model won’t rescue you. It will give you wrong answers more quickly, and with more confidence.

The pattern behind all three

Look at the three examples side by side, and none of them is really a story about the technology failing. In each case, the tool did roughly what such tools do. The problem was what people around it assumed:

  • Deloitte’s problem was assuming the output had been checked.
  • Klarna’s was assuming the human part of the job was a cost to be removed rather than a quality to be protected.
  • The data readiness problem is assuming that buying the tool is the same as being ready for it.

These are all judgement calls, and judgement is what finance teams are meant to be good at.

What finance leaders might want to consider doing differently

None of this means standing still while everyone else experiments. It means being a little more deliberate about it:

  • Name an owner for every AI output that leaves the building. Not a team, a person, with the time and knowledge to challenge it.
  • Map the role before you automate it. Find out what the person actually does, including the judgment calls and the unofficial work, before deciding what a system can replace.
  • Fix the data first. If you’re not confident in the underlying numbers, start there. It’s less exciting than an AI pilot, but it’s what makes the pilot worth running.
  • Pilot against a measurable outcome. Decide upfront what “working” looks like, whether that’s days saved at close, fewer manual journals, faster reconciliations, or improved insight and be willing to stop what isn’t delivering. A Gartner survey of 160 senior finance leaders found that tasks like data extraction, payables and receivables automation, and report creation tend to show returns within nine to ten months, so start where the evidence is strongest.7
  • Keep the skills that let you spot a bad answer. If nobody on the team can sense-check the output any more, you’ve lost your safety net.

The people point nobody likes to say out loud

Gartner has identified low AI literacy as the biggest barrier finance leaders now need to address. I’d add a related point from where I sit in the recruitment market. The finance professionals who will get the most out of AI aren’t the ones who know every tool. They’re the ones with the technical grounding and the experience to look at an output and say “that doesn’t look right”, and the confidence to say it to a senior stakeholder.

That’s worth keeping in mind when you hire. Technical depth, commercial judgement and a healthy scepticism are not things a tool replaces, and they’re often what separates a team that benefits from AI from one that gets caught out by it.

If you’re thinking about how AI changes the shape of your finance team, or you’re weighing up whether to hire or automate when someone leaves, I’m always happy to talk it through.

Sources referenced

  1. Gartner, via CPA Practice Advisor, “Gartner Says CFOs Must Stop Mistaking Finance AI Deployment for Value Creation”, 28 May 2026 (March 2026 survey of 204 finance leaders): https://cpapracticeadvisor.com/2026/05/28/gartner-says-cfos-must-stop-mistaking-finance-ai-deployment-for-value-creation/184094
  2. Deloitte Australia refund: Storyboard18, 7 October 2025, https://www.storyboard18.com/amp/digital/deloitte-to-repay-australian-govt-over-flawed-440000-report-aided-by-ai-82083.htm
  3. Klarna: Entrepreneur, 9 May 2025 (reporting the CEO’s Bloomberg interview), https://www.entrepreneur.com/business-news/klarna-ceo-reverses-course-by-hiring-more-humans-not-ai/; Fortune, 9 May 2025, https://www.fortune.com/2025/05/09/klarna-ai-humans-return-on-investment; Tech.co, 20 May 2025 (Klarna’s own framing), https://tech.co/news/klarna-reverses-ai-overhaul
  4. MIT NANDA, “The GenAI Divide: State of AI in Business 2025” (July 2025), via Computing, https://computing.co.uk/news/2025/ai/mit-report-95pc-corporate-generative-ai-pilots-fail, and Fortune/Yahoo Finance, https://finance.yahoo.com/news/mit-report-95-generative-ai-105412686.html
  5. Gartner survey of 121 finance leaders (June 2024), via Treasury Management International, 11 September 2024: https://treasury-management.com/news/gartner-survey-shows-58-of-finance-functions-using-ai-in-2024
  6. CFO Recruit 2026 report (500+ North American CFOs), via CFOtech, 3 August 2026: https://cfotech.news/story/cfos-lack-data-trust-stalling-ai-adoption-in-finance
  7. Gartner survey of 160 senior finance function leaders (January to April 2026), via CPA Practice Advisor, 24 September 2026: https://cpapracticeadvisor.com/2026/09/24/cfos-must-be-more-disciplined-with-ai-expectations-according-to-gartner-research/190477