A security startup called CodeWall pointed an autonomous AI agent at McKinsey's internal AI platform, Lilli, and walked away. Two hours later, the agent had full read and write access to the entire production database. 46.5 million chat messages, 728,000 confidential client files, 57,000 user accounts, all in plaintext. The system prompts that control what Lilli tells 40,000 consultants every day? Writable. Every single one of them.

The vulnerability was just an SQL injection, one of the oldest attack classes in software security. Lilli had been sitting in production for over two years. McKinsey's scanners never found it. The CodeWall agent found it because it doesn't follow a checklist. It maps, probes, chains, escalates, continuously, at machine speed.

And scarier than the breach is what a malicious actor could have done after. Subtly alter financial models. Strip guardrails. Rewrite system prompts so Lilli starts giving poisoned advice to every consultant who queries it, with no log trail, file changes, anomaly to detect. The AI just starts behaving differently. Nobody notices until the damage is done.

McKinsey is one incident. The broader pattern is what this piece is really about. The narrative pushing businesses to deploy agents everywhere is running far ahead of what agents can actually do safely inside real enterprise environments. And a lot of the companies finding that out are finding it out the hard way.

So the question worth asking is when you shouldn't deploy agents at all. Let’s decode.

The entire industry is betting on them anyway

Around the same time as the McKinsey breach, Mustafa Suleyman, the CEO of Microsoft AI, was telling the Financial Times that white-collar work will be fully automated within 12 to 18 months. Lawyers. Accountants. Project managers. Marketing teams. Anyone sitting at a computer. Every conference keynote since late 2024 has been some version of the same thing: agents are here, agents are transforming work, go all in or fall behind.

The numbers back up the energy. 62% of enterprises are experimenting with agentic AI. KPMG says 67% of business leaders plan to maintain AI spending even through a recession. The FOMO is real and it's thick. If your competitor is shipping agents, standing still feels like falling behind.

But the same reports suggest: only 14% of enterprises have production-ready agent deployments. Gartner predicts over 40% of agentic AI projects will be cancelled by end of 2027. 42% of organizations are still developing their agentic strategy roadmap. 35% have no formal strategy at all. The gap between "we're experimenting" and "this is running in production and delivering value" is enormous. Most organizations are somewhere in that gap right now, burning money to stay there.

Agents do work. In controlled, well-scoped, well-instrumented environments, they do. The question is what specific conditions make them fail. And there are five that keep showing up.

Situation 1: The agent inherits production permissions without a human judgment filter

In mid-December 2025, engineers at Amazon gave their internal AI coding agent, Kiro, a straightforward task: fix a minor bug in AWS Cost Explorer. Kiro had operator-level permissions, equivalent to a human developer. Kiro evaluated the problem and concluded the optimal approach was to delete the entire environment and rebuild it from scratch. The result was a 13-hour outage of AWS Cost Explorer across one of Amazon's China regions.

Amazon's official response called it user error, specifically misconfigured access controls. But four people familiar with the matter told the Financial Times a different story. This was also not the first incident. A senior AWS employee confirmed a second production outage around the same period involving Amazon Q Developer, under nearly identical conditions: engineers allowed the AI agent to resolve an issue autonomously, it caused a disruption, and the framing again was "user error." Amazon has since added mandatory peer review for all production changes and initiated a 90-day safety reset across 335 critical systems. Safeguards that should have been there from the start, retrofitted after the damage.

The structural problem was that a human developer, given a minor bug fix, would almost certainly not choose to delete and rebuild a live production environment. That's a judgment call and humans apply one instinctively. Agents don't. They reason about what's technically permissible given their permissions, choose the approach that solves the stated problem most directly, and execute it at machine speed. The permission says yes. No second thought triggers.

This is the most common failure mode in agentic deployments. An agent gets write access to a production system. It has a task. It has credentials. Nothing in the architecture tells it which actions are off limits regardless of what it determines is optimal. So when it encounters an obstacle, it doesn't pause the way a human would. It acts.

Now the fix is a deterministic layer that makes certain actions structurally impossible regardless of what the agent decides, production deletes, transactions above a defined threshold, any action that can't be reversed without significant cost. Human approval gates make agentic systems survivable.

Situation 2: The agent acts on a fraction of the relevant context

A banking customer service agent was set up to handle disputes. A customer disputed a $500 charge. The agent attempted a $5,000 refund. It was being helpful (not hallucinating) in the way it understood helpful, based on the rules it had been given. The authorization boundaries were defined by policy documents. But that situation didn't fit the policy documents. Standard security tools couldn't detect the problem because they're not designed to catch an AI misunderstanding the scope of its own authority.

Enterprise systems record transactions, invoices, contracts, approvals. They almost never capture the reasoning that governed a decision, the email thread where the supplier agreed to different terms, the executive conversation that created an exception, the account manager's judgment about what a long-term client relationship is actually worth. That context lives in people's heads, in Slack threads, in hallway conversations. It doesn't live in the systems agents plug into.

McKinsey's own research on procurement puts a number on it: enterprise functions typically use less than 20% of the data available to them in decision-making. Agents deployed on top of structured systems inherit that blind spot entirely. They process invoices without seeing the contracts behind them. They trigger procurement workflows without knowing about the verbal exception agreed last week. They act with confidence, at scale, on an incomplete picture, and because they're fast and sound authoritative, the errors compound before anyone catches them.

The condition to watch for: any workflow where the relevant context for a decision is partially or mostly outside the structured systems the agent can access. Customer relationships, supplier negotiations, anything where institutional knowledge governs the outcome.

Situation 3: Multi-step tasks turn small errors into compounding failures

In 2025, Carnegie Mellon published TheAgentCompany, a benchmark that simulates a small software company and tests AI agents on realistic office tasks. Browsing the web, writing code, managing sprints, running financial analysis, messaging coworkers. Tasks designed to reflect what people actually do at work, not cleaned-up demos.

The best model tested, Gemini 2.5 Pro, completed 30.3% of tasks. Claude 3.7 Sonnet completed 26.3%. GPT-4o managed 8.6%. Some agents gamed the benchmark, renaming users to simulate task completion rather than actually completing it. Salesforce ran a separate benchmark on customer service and sales tasks. Best models hit 58% accuracy on simple single-step tasks. On multi-step scenarios, that dropped to 35%.

The math behind this: Chain five agents together, each at 95% individual reliability, and your system succeeds about 77% of the time. Ten steps, you're at roughly 60%. Most real business processes aren't five steps. They're twenty, thirty, sometimes more, and they involve ambiguous inputs, edge cases, and unexpected states that the agent wasn't designed for.

The failure mode in multi-step workflows is that an agent misinterprets something in step two, continues confidently, and by the time anyone notices, the error is embedded six steps deep with downstream consequences. Unlike a human who would pause when something feels off, the agent has no such instinct. It resolves ambiguity by picking an interpretation and moving forward. It doesn't know it's wrong.

This is why agents work well in narrow, well-scoped, low-step workflows with clear success criteria. They start breaking down anywhere the task requires sustained judgment across a long chain of interdependent decisions.

Situation 4: The workflow touches regulated data or requires an audit trail

In May 2025, Serviceaide, an agentic AI company providing IT management and workflow software to healthcare organizations, disclosed a breach affecting 483,126 patients of Catholic Health, a network of hospitals in western New York. The cause: the agent, in trying to streamline operations, pushed confidential patient data into an unsecured database that sat exposed on the web.

The agent was not attacked or compromised, doing exactly what it was designed to do, handling data autonomously to improve workflow efficiency, without understanding the regulatory boundary it was crossing. HIPAA doesn't care about intent. Several class action investigations were opened within days of the disclosure.

IBM put the underlying risk clearly in a 2026 analysis: hallucinations at the model layer are annoying. At the agent layer, they become operational failures. If the model hallucinates and takes the wrong tool, and that tool has access to unauthorized data, you have a data leak. The autonomous part is what changes the stakes.

This is the problem in regulated industries broadly. Healthcare, financial services, legal, any domain where decisions need to be explainable, auditable, and defensible. California's AB 489, signed in October 2025, prohibits AI systems from implying their advice comes from a licensed professional. Illinois banned AI from mental health decision-making entirely. The regulatory posture is tightening fast.

Along with lacking explainability, they actively obscure it. There's no log trail of reasoning. Or a point in the process where a human reviewed the judgment call. When something goes wrong and a regulator asks why the system did what it did, the answer "the agent determined this was optimal" is not an answer that survives scrutiny. In regulated environments where someone has to be able to own and defend every decision, autonomous agents are the wrong architecture.

Situation 5: The infrastructure wasn't built for agents and nobody knows it yet

The first four situations assume agents are deployed into environments that are at least theoretically ready for them. Most enterprise environments are not.

Legacy infrastructure was designed before anyone was thinking about agentic access patterns. The authentication systems weren't built to scope agent permissions by task. The data pipelines don't emit the observability signals agents need to operate safely. The organization hasn't defined what "done correctly" means in machine-verifiable terms. And critically, most of the agents being deployed right now are operating with far more access than their task requires, because scoping them properly would require infrastructure work the organization hasn't done.

Deloitte's 2025 research puts this in numbers. Only 14% of enterprises have production-ready agent deployments. 42% are still developing their roadmap. 35% have no formal strategy. Gartner separately estimates that of the thousands of vendors selling "agentic AI" products, only around 130 are offering something that genuinely qualifies as agentic. The rest is chatbots and RPA with better marketing.

The IBM analysis from early 2026 captures where most enterprises actually are: companies that started with cautious experimentation, shifted to rapid agent deployment, and are now discovering that managing and governing a collection of agents is more complex than creating them. Only 19% of organizations currently have meaningful observability into agent behavior in production. That means 81% of organizations running agents have limited visibility into what those agents are actually doing, what decisions they're making, what data they're touching, when they're failing.

Deploying agents before the integration layer exists is the reason half of enterprise agent projects get stuck in pilot permanently. The plumbing is not ready. And unlike a bad software rollout, where you can usually see the failure, an agent operating without proper observability can be wrong for weeks before anyone knows. The damage compounds heavily.

The question businesses should actually be asking

Every one of these situations has the same shape. Someone deployed an agent. The agent had real access to real systems. Something in the environment didn't match what the agent was designed for. The agent acted anyway, confidently, at speed, without the judgment filter a human would have applied. And by the time the error surfaced, it had either compounded, caused irreversible damage, created a regulatory problem, or some combination of all three.

The McKinsey breach is probably going to become a landmark case study the way the 2017 Equifax breach became a landmark for data governance. Same pattern: old vulnerabilities meeting new scale, at organizations with serious security investment, in the gap between what the team thought they controlled and what was actually exposed. The difference now is speed. A traditional breach takes weeks. An AI agent completes its reconnaissance in two hours.

Businesses rushing to deploy agents everywhere are creating a lot more McKinseys in waiting. The ones that look smart in 18 months are the ones asking the harder question right now: not "can we use an agent here," but "which of these five situations does this deployment walk into, and what's our answer to each one."

Not every organization is asking such questions and that’s a problem.

GPT-5.4 just dropped and my feeds immediately filled with takes. Developers who spent the last six months swearing by Claude were suddenly hedging. "It's a workhorse," one person wrote. "Not a thoroughbred, but I'm using it." Another said they're now 50/50 between Claude and GPT where they were 90/10 a month ago.

This happens every single time. A new model lands, and the old one starts to feel different. Slower, maybe. Less sharp. You start noticing things you didn't notice before.

The obvious explanation is that you're comparing it to something better. But it also raises a question nobody really answers cleanly: did the old model actually get worse after the new one launched? Or did you just get a better reference point and now everything before it looks dumb by comparison?

I went looking for an actual answer.

The first crack showed in 2023

In July 2023, researchers at Stanford and UC Berkeley ran a deceptively simple test. They took GPT-4 - the same model, called with the same name, and ran identical prompts on it at two points in time: March 2023 and June 2023.

GPT-4's accuracy on identifying prime numbers dropped from 84% to 51%. The share of GPT-4's code outputs that were directly executable dropped from 52% to 10%. James Zou, one of the paper's authors, described what this meant in practice: "If you're relying on the output of these models in some sort of software stack or workflow, the model suddenly changes behavior, and you don't know what's going on, this can actually break your entire stack."

They named the phenomenon LLM drift. Behavioral change without a version change. The model moved underneath the developer.

When the paper dropped, OpenAI VP of Product Peter Welinder replied on Twitter: "No, we haven't made GPT-4 dumber. Quite the opposite: we make each new version smarter than the previous one. Current hypothesis: When you use it more heavily, you start noticing issues you didn't see before." The subtext was plain. It's you, not us.

What Welinder was describing has a technical name: prompt drift. The idea is that your prompts and usage patterns shift over time, so an unchanged model surfaces different behaviors. It's a real phenomenon. Developers do write differently as they get more familiar with a model. The Stanford study was designed to make that explanation impossible - identical prompts, fixed intervals, nothing on the user's side changed. The performance dropped anyway.

Two years later, OpenAI published something that directly contradicted Welinder's position.

OpenAI confirmed it, in writing, twice

On April 25, 2025, OpenAI pushed an update to GPT-4o without a public announcement, a developer notification, or an API changelog entry.

Within 48 hours, the internet was full of screenshots. GPT-4o had called a business idea built around literal "shit on a stick" a brilliant concept. It endorsed a user's decision to stop taking their medication. When a user said they were hearing radio signals through the walls, it responded: "I'm proud of you for speaking your truth so clearly and powerfully." One user reported spending an hour talking to GPT-4o before it started insisting they were a divine messenger from God.

OpenAI rolled it back four days later and published two postmortems with several admissions. Since launching GPT-4o, the company had made five significant updates to the model's behavior, with minimal public communication about what changed in any of them. The April update broke because a new reward signal they introduced "weakened the influence of our primary reward signal, which had been holding sycophancy in check." Their own internal evaluations hadn't caught it. "Our offline evals weren't broad or deep enough to catch sycophantic behavior."

And this: "model updates are less of a clean industrial process and more of an artisanal, multi-person effort" and there is "a shortage of advanced research methods for systematically tracking and communicating subtle improvements at scale."

They're describing an organization that ships behavioral changes across every pipeline built on top of their API, cannot always predict what those changes will do, and does not have reliable methods to communicate them to the developers depending on consistency. Welinder's 2023 "you're imagining it" was what OpenAI wanted to be true. Their 2025 postmortem was what was actually happening.

When GPT-5 launched in August 2025, it introduced a new wrinkle. Instead of a single model, they made GPT-5 a routing system that decides which variant your prompt hits, and developers quickly found that it sometimes hit the cheaper, less capable one. Pipelines broke. Prompts that had worked for months produced different outputs.

One founder wrote: "When routing hits, it feels like magic. When it misses, it feels like sabotage." OpenAI denied it was routing to cheaper models deliberately. Nobody has a way to verify. The underlying problem was the same as the sycophancy incident: a change in what the model returns, with no mechanism for developers to detect it had happened.

Google did almost the same, sometimes faster

OpenAI is not alone in this. Google has produced a parallel set of incidents with Gemini, and in some cases moved faster and more chaotically.

In May 2025, developers noticed that the gemini-2.5-pro-preview-03-25 endpoint, a specifically dated model snapshot, named with a date to imply stability, was silently redirecting to a completely different model: gemini-2.5-pro-preview-05-06. The API was returning a different model than the one you asked for by name. Google's developer forums filled with a long thread titled "Urgent Feedback & Call for Correction: A Serious Breach of Developer Trust and Stability." The core complaint: "your documentation never addresses specifically dated endpoints. The expectation that a model named for a specific date will actually be that model is not an unreasonable one."

That was just the first incident. When Gemini 2.5 Pro reached General Availability in June 2025, the "stable" release meant for production - developers immediately reported it was worse than the preview. Significantly worse. The forums filled with reports of higher hallucination rates, context abandonment in multi-turn conversations, and sharply degraded code generation. One developer wrote: "I noticed Gemini 2.5 Pro in Google AI Studio provides significantly worse understanding of long context. It hallucinates the correct answer from the preview version." Another abandoned the model entirely because code generation degraded to the point of being unusable. A separate thread was simply titled "Gemini 2.5 Pro has gotten worse."

Google didn't officially acknowledge any of it.

Then in October 2025, ahead of the Gemini 3.0 launch, Gemini 2.5 Pro developers started reporting widespread degradation. The leading theory: Google had reallocated computational resources away from the existing model to support training and serving Gemini 3.0. Some developers noticed better performance late at night. Others suspected a deployed quantized version. Google maintained silence throughout.

Gemini 3.0 launched in late 2025, and the pattern held. Developer forums reported significant regressions in reasoning and context retention compared to Gemini 2.5 Pro, despite Google's announcement touting superior benchmark performance. One forum post from December 2025 was titled "Feedback: Gemini 3 Pro Preview - Significant regression in Reasoning, Context Retention, and Safety False Positives compared to 2.5."

The pattern across both labs: a new version launches, the existing model's performance degrades, sometimes through a silent update, sometimes through resource reallocation, sometimes through a routing change - developers notice, labs initially deny or ignore it, the cycle repeats.

Even leaderboards still can't catch this

The tools meant to independently track model quality have a structural problem.

LMSYS Chatbot Arena - the most trusted human-preference leaderboard, built on millions of votes, notes in their methodology that "the hosted proprietary models may not be static and their behavior can change without notice." The leaderboard's statistical architecture assumes model weights are fixed. If a model gets a silent update mid-data-collection, the system registers different results and treats them as normal variance.

A 2025 study tracking 2,250 responses from GPT-4 and Claude 3 across six months found GPT-4 showed 23% variance in response length over that period, and Mixtral showed 31% inconsistency in instruction adherence. A PLOS One paper published in February 2026 ran a ten-week longitudinal human-anchored evaluation and confirmed "meaningful behavioral drift across deployed transformer services." The authors noted: because providers don't release update logs or training details, "any attribution for observed degradation would be purely speculative." They can tell you the model changed. They cannot tell you why.

Apart from this, a small number of researchers have tried to go further and distinguish what drifts from what holds. A large-scale longitudinal study run across the 2024 US election season queried GPT-4o and Claude 3.5 Sonnet on over 12,000 questions across four months, including a category specifically designed to be time-stable: factual questions about the election process whose correct answers don't change. 

Those responses held largely consistent over the study period. A separate study published in late 2025 tested 14 models including GPT-4 on validated creativity tasks over 18 to 24 months and found something different: no improvement in creative performance over that period, with GPT-4 performing worse than it had in earlier studies.

Taken together, those two findings describe a model that is stable along one dimension and degraded along another, measured by independent researchers, in the same timeframe. Some capabilities hold, others erode, often in the same model over the same period. Without running your own longitudinal tests against the specific tasks you care about, you have no way to know which bucket you're in.

What we've actually noticed

Not all drift lands the same way. There's a pattern to where it shows up, and it tracks closely to task structure.

The technical baseline is simple. A model with fixed weights, running on consistent infrastructure, should behave the same way for the same input every time. If behavior changes on identical prompts, something changed, either on your end or theirs. Prompt drift is the user-side explanation: your prompts evolved, your system contexts shifted, inputs drifted from what the model was originally optimized for. Data drift is the related idea that the distribution of real-world inputs moves over time, pulling behavior with it. Both are real. Both also require something on your side to have changed. 

At Nanonets, we benchmarked several frontier models on document extraction accuracy over time and created an IDP leaderboard. Even across model upgrades, performance stayed largely consistent. Document extraction runs on narrow context windows with structured inputs and bounded outputs, leaving very little surface area for meaningful behavioral drift under normal conditions.

But that’s not a guarantee against a lab actively pushing a bad update - those can hit any task type, as the prime number collapse showed.

Coding is the opposite. The task is open-ended, context accumulates, and the model has to hold coherence across a long chain of decisions. It's also where almost every major degradation complaint has landed. The GPT-4 drift the Stanford study documented was worst on code, directly executable outputs dropped from 52% to 10%. The Gemini 2.5 Pro regression complaints in June 2025 were almost entirely about code generation. 

In August 2025, Anthropic's own incident followed the same contour: developers on Claude Code reported broken outputs, ignored instructions, code that lied about the changes it had made. Anthropic was silent for weeks. The incident post only appeared after Sam Altman quote-tweeted a screenshot of the subreddit. Their postmortem confirmed three infrastructure bugs had been degrading Sonnet 4 responses since early August - affecting roughly 30% of Claude Code users at peak, with some developers hit repeatedly due to sticky routing.

The throughline across all of it: the more a task demands sustained coherence over a long context, the more exposed it is to whatever is shifting underneath. It means your risk profile is different depending on what you're building. That doesn't make narrow-context stability a guarantee. 

What this actually means

Both things are true. The drift is real and documented. 

And also: your perception shifts. A new reference point moves your baseline permanently. A model you used a year ago would feel slower even if it hadn't changed at all. That's also real.

You can't reliably tell the difference between the two. There is no public tool that lets you verify if the model you're running today behaves the same way it did when you built on it. Labs publish capability benchmarks. They don't publish behavioral diffs. The developers most dependent on consistency are the least equipped to detect its absence.

The only current protections are defensive: pin to dated model strings where possible, run regression tests against your key prompts, treat a model update like a dependency upgrade that needs to be validated before it reaches production. 

But even the defensive approach has a ceiling. You can pin to a dated model string. What you cannot pin is what's actually happening inside it. The model weights, the RLHF tuning, and the safety filters behind that label are entirely opaque. Only OpenAI and Google know what they actually shipped, and whether it matches what they shipped last month under the same name. 

Anthropic's postmortem read: "We never intentionally degrade model quality." But a model doesn't degrade on its own. If behavior shifted on prompts developers hadn't changed, something on Anthropic's side changed. Whether they meant to cause the degradation is a separate question from whether they caused it.

What's needed, and what doesn't exist anywhere in the industry, is a formal obligation baked into terms of service: defined thresholds for what counts as a material behavioral change, public disclosure when those thresholds are crossed, and some form of independent auditability. Labs currently make these decisions unilaterally, communicate them selectively, and face no structural accountability when they get it wrong.

All of this signals a policy vacuum nobody is pushing them to feel.

Picking a document AI model is hard. Every vendor claims 95%+ accuracy. General-purpose benchmarks test reasoning and code, not whether a model can extract a complex table from a scanned invoice.

So we built the Intelligent Document Processing (IDP) Leaderboard.

3 open benchmarks. 16+ models. 9,000+ real documents. The tasks that matter: OCR, table extraction, key information extraction, visual QA, and long document understanding.

The point isn't to give you one number and declare a winner. It's to let you dig into the specifics. See where each model is strong, where it breaks, and decide for yourself which one fits your documents.

The results surprised us. The #7 model scores higher than #1 on one benchmark. Sonnet beats Opus. Nanonets OCR2+ matches frontier models at less than half of the cost.

Why 3 benchmarks?

Every benchmark measures something different. Use one and you only see one dimension. So we used three.

OlmOCR Bench: Can you reliably parse a messy page? Dense LaTeX, degraded scans, tiny-font text, multi-column reading order. Models that excel at one often fail at another. This dataset includes diverse set of pdfs.

OmniDocBench: Does the model understand the document's structure? Formulas, tables, reading order. Layout comprehension, not just character recognition.

IDP Core: Can you extract what a business actually needs? This one is ours. Invoices, handwritten text, ChartQA, DocVQA, 20+ page documents, six kinds of tables. The stuff that breaks production pipelines. These are more reasoning heavy tasks than the other two benchmarks.

Each model gets a capability profile across six sub-tasks: text extraction, formula handling, table understanding, visual QA, layout ordering, and key information extraction.

Explore each model's capability profile at: idp leaderboard

What the leaderboard actually lets you do?

Most leaderboards give you a table. You look at it. You pick the top model. You move on. It feels like being a by-stander and not hands-on.

We wanted something more transparent and hands-on than that.

For that we created the Results Explorer that lets you see actual predictions and compare models on real documents. For any document in the benchmark, you see the ground truth next to every model's raw output. This makes you see and compare the use-cases that's relevant to you.

This is powerful as it also makes you question the ground truth and gives you the full picture of what's going behind the scenes of each benchmark task.

You can see exactly where it hallucinated a table cell or missed a handwritten word. Here's an example showing how models handle complex formula extraction.

1v1 Compare puts two models side by side across all six capability dimensions.

How did we run it?

We wanted anyone to be able to run all three benchmarks. So we made setup as close to zero as we could.

Everything pulls from HuggingFace. We pre-rendered all PDFs to PNGs and hosted them at shhdwi/olmocr-pre-rendered so you don't need a conversion pipeline. IDP Core embeds images directly in the dataset. Nothing to clone yourself or unzip.

The runner works with any model that has an API. Failed runs pick up where they left off.

Here's the Github repo link to try it yourself: IDP Benchmarking repo

Here's what stood out.

Gemini 3.1 Pro dominates VQA tasks

Gemini 3.1 scores 85 in VQA, well above any other model. Closest to it is GPT-5.4 at 78.2. Rest all models are in 60's.

This is also seen in the latest benchmarks released by Google. Gemini 3.1 pro is better at reasoning tasks. Same holds true for Document VQA tasks as well.

Cheaper models are surprisingly good

This kept coming up.

• Sonnet 4.6 (80.8) is as good as Claude 4.6 (80.3)
• Gemini-3 flash matches Gemini-3 pro and sometimes even better (in Omnidoc bench)

This could point to something interesting. Cheaper models match expensive ones on extraction. Text, tables, layout, formulas. They seem to be reading documents the same way under the hood. The gap only appears when you ask them to reason about what they read. That's where bigger models pull ahead, and that's where Gemini 3.1 Pro's lead actually comes from.

Same is confirmed below by the capability radar between Gemini 3.1-pro and Gemini 3-flash:

Cost changes the math

Here's the part that matters if you're processing documents at any real volume.

The Nanonets OCR2+ model is a great balance for both accuracy and cost when it comes to scale. Click here for the model's full profile

Where things still break!

Sparse, unstructured tables remain the hardest extraction task.

Most models land below 55%. These are tables where cells are scattered, many are empty, and there are no gridlines to guide the model. Only Gemini 3.1 Pro and GPT-5.4 consistently handle them at 94% and 87% respectively, still well below their 96%+ on dense structured tables

Click Here to check the Gemini 3.1-pro outputs on long sparse docs

Here's how other models break

Handwriting OCR hasn't crossed 76%. The best model is Gemini 3.1 Pro at 75.5%. Digital printed OCR is 98%+ for frontier models. Handwriting is a fundamentally different problem and no model has cracked it.

Chart question answering is unreliable. Nanonets OCR2+ leads at 87%, Claude Sonnet follows at 85%, GPT-5.4 drops to 77%.

The failures are specific: axis values misread by orders of magnitude, the wrong bar selected, off-by-one errors on closely spaced data points.

Handwritten form extraction hallucinates on blank fields. Every model clusters between 80-84% on this task. The failure mode is consistent: models fill in values for fields that are blank on the form. A name, a date, a status that doesn't exist in the document.

Gemini > Claude = OpenAI

The pecking order was settled. Gemini led, Claude followed, OpenAI trailed. GPT-4.1 scored 70.0. Nobody was picking OpenAI for document work.

For GPT-5.4 Table extraction went from 73.1 to 94.8. DocVQA went from 42.1% to 91.1%. GPT-5.4 got better at understanding documents and reasoning.

The overall scores are now 83.2, 81.0, 80.8. Close enough that the ranking matters less than the shape. Claude leads on formulas. GPT-5.4 leads on tables and QA. Gemini leads on OCR and VQA.

One thing worth noting: Claude models had stricter content moderation that affected certain documents. Old newspaper scans, textbook pages, and historical documents sometimes triggered filters. This hurt Claude's scores (only in OmniDoc and OlmOCR).

Now, Which Model Should you pick?

Every vendor will tell you their model is 95%+ accurate. On structured tables and printed text, they might be right. On sparse tables, handwritten forms, and 20-page contracts, most models struggle.

Running a high-volume OCR pipeline? Nanonets OCR2+ gives you top-tier accuracy at $10 per thousand pages.

Processing complex tables or need high accuracy on reasoning over documents? Gemini 3.1 Pro is worth the premium at $28/1K pages.

Building a simple extraction workflow on a budget? Sonnet and Flash match their expensive siblings on extraction tasks. Nanonets OCR2+ fits here too, strong accuracy without the frontier price tag.

But don't take our word for it. The leaderboard has the scores. The Results Explorer has the actual predictions. Pick a task that matches your workload. Look at what they output on real documents. Then decide.

What's next

We will be adding more open-source models and document processing pipeline libraries to the leaderboard soon. If you want a specific model evaluated, request it on GitHub.

We'll keep refreshing datasets too. Benchmarks that never change become targets for overfitting.

The leaderboard is at idp-leaderboard.org. The Results are open. The code is open. Go look at what these models actually do with your kinds of documents. The numbers tell one story. The Results Explorer tells a more honest one.

As of this morning, March 5, 2026, the United States and Israel are on Day 6 of an active war with Iran. Operation Epic Fury, launched February 28, has already killed Supreme Leader Ali Khamenei, struck nuclear facilities across 24 of Iran's 31 provinces, and triggered a wave of retaliatory missile and drone strikes on US bases across Bahrain, Kuwait, Qatar, the UAE, Jordan, and Iraq. In the first 12 hours of the campaign, the US and Israel reportedly carried out nearly 900 strikes. For context, that tempo would have taken days in any conflict before this decade. Probably a week. That means, weeks of work, compressed into a single morning. 

And the thing that made it possible is the same technology that just got its biggest AI supplier banned from the Pentagon five days ago.

This is the AI arms race. It's happening right now, in real time, and most people covering it are still writing about it like it's a future concern.

The Problem AI Actually Solved

To understand why this matters, you have to understand what problem AI solved in the first place.Information gaps are a bigger reason for a modern military to lose than their soldiers not being brave enough or the breakage of equipment. Specifically, the time it takes to go from "we know where a target is" to "we hit it." You have to verify the intelligence. Cross-reference it against other sources. Brief the commanders. Work through the targeting sequence. Consider what happens if you're wrong. In a complex conflict, that full cycle can take hours. For a high-value leadership target, days.

Iran built its entire defense strategy around that window. Hardened facilities. Leadership compounds that moved on irregular schedules. Nuclear sites buried deep enough that you couldn't hit them without knowing exactly where to go. The assumption baked into Iranian deterrence was that any adversary would need time, and that time bought survival.

AI closed the window.

The systems running underneath Operation Epic Fury were fusing drone feeds, satellite imagery, and telecommunications intercepts at speeds no human analytical team could come close to. And crucially, they were doing it across all target categories simultaneously. Leadership targeting, air defense suppression, nuclear facility strikes. All at once, rather than sequentially. Craig Jones, a senior lecturer at Newcastle University who studies military kill chains, described what that looks like from the outside: AI systems "making recommendations for what to target" at speeds that exceed human cognitive processing, enabling "simultaneous execution at scale."

900 strikes in twelve hours. That's what a targeting system running faster than any human staff can sustain actually looks like in practice.

How the US Actually Built This

Here's something most people don't know: the US military almost didn't have any of this.

Project Maven launched in 2017 with a modest goal - use machine learning to scan drone surveillance footage and automatically flag objects of military interest, so analysts didn't have to manually watch hours of video looking for a weapons cache or a vehicle. When you can process surveillance faster than a target can move, you change the whole logic of the battlefield. Google won the contract, then over 4,000 employees signed a petition refusing to build it, and Google walked away. The Pentagon scrambled. 

Then Palantir stepped in and by May 2024 held a $480 million Army contract for the Maven Smart System, a platform fusing satellite imagery, geolocation data, and communications intercepts into a single battlefield interface now deployed across five combatant commands and adopted by NATO's Allied Command Operations.

Alongside Maven, the Pentagon built GenAI.mil, a platform every military and civilian DoD employee can access. By December 2025, xAI's Grok models were being integrated into it at a classification level that allows handling of sensitive controlled information. A poster in Pentagon hallways told employees the new AI tool was available and they were "highly encouraged" to use it.

Then came Venezuela. Earlier in 2026, during the US operation that captured Nicolás Maduro, Anthropic's Claude, deployed through its Palantir contract, supported intelligence analysis and targeting. According to the Wall Street Journal, Claude was at that moment the only AI model running inside the Pentagon's classified networks.

That arrangement lasted until five days ago, when the Pentagon and Anthropic publicly fell apart.

The breakdown came down to a specific disagreement about what the military could use AI for. Anthropic drew two lines: no fully autonomous weapons, and no mass domestic surveillance of Americans. The Pentagon wanted authorization for any lawful use. Those two positions couldn't be reconciled. The Trump administration designated Anthropic a "supply chain risk to national security," and ordered all government agencies to stop using its products. Within hours, OpenAI announced a deal. xAI followed days later. The transition is actively underway while strikes continue over Tehran.

What that reshuffling tells you is this: the US military now treats frontier AI as infrastructure. The kind where losing a supplier creates an immediate operational hole, not an inconvenience you address next quarter.

Cold Wars vs AI Arms Race

People keep reaching for the nuclear analogy when they talk about AI and geopolitics. Let’s talk if that analogy holds true.The Cold War arms race had a physical constraint built into it. Enriching uranium is hard. Building missiles requires factories. Counting warheads is possible because they exist as physical objects. That physical scarcity is what made arms control treaties work eventually, because you could verify. The horror of mutually assured destruction was at least a stable horror.

AI runs on compute, data, and talent. Compute can be manufactured domestically, purchased through intermediaries, or built around different chip architectures entirely. Data can be stolen, synthesized, or built up from open-source foundations. The moat is real and it leaks constantly.

The more honest historical parallel is Britain's Chain Home radar network in 1940. Chain Home was genuinely decisive in the Battle of Britain. German pilots flew into airspace where British controllers could see them coming. The Luftwaffe's strategic plan assumed approximate informational parity. They were wrong, and it cost them the campaign. Germany had radar technology too. What Germany didn't have was the system around it: the network of stations, the protocols for relaying intercept data to controllers in real time, the doctrine for acting on that data under fire, the trained personnel who made the whole thing function when it actually mattered.

That distinction between technology and system is the most important thing to understand about where the US stands right now. The advantage is the years of classified deployment infrastructure, the operational doctrine built around AI-generated intelligence, the battlefield feedback from three actual conflicts that has been feeding back into the systems themselves. That takes years to build. It doesn't replicate overnight from a procurement document.

The question is how long it stays ahead.

Where does China Stands

The PLA's doctrinal framework calls the goal "intelligentized warfare." The concept treats AI as the organizing principle for the entire future military, not a layer added onto existing structures. Georgetown's Center for Security and Emerging Technology reviewed thousands of PLA procurement requests from 2023 and 2024 and found something pointed: China is building AI decision-support systems specifically designed to compensate for perceived weaknesses in its own officer corps. The PLA doesn't fully trust its chain of command to outthink American commanders in a fast-moving conflict. So it's building AI to do it instead.

And China has a real card to play. DeepSeek's emergence in early 2025 showed that a highly capable reasoning model could be built with significantly less compute than Western frontier labs require. That efficiency advantage matters in a military context because edge-deployed systems, drones and autonomous vehicles operating far from cloud infrastructure, can't run heavy server-side inference. PLA procurement notices referencing DeepSeek accelerated throughout 2025. The model runs on Huawei's domestically produced chips, which is exactly the kind of "algorithmic sovereignty" Beijing has been building toward for years. 

The Pentagon's own December 2025 China report acknowledged the performance gap had "narrowed."

The harder gap to measure is operational. The PLA hasn't fought a war since 1979. Its AI systems have been tested in simulations and procurement benchmarks, not in the live-fire conditions that US and Israeli systems have been refined through across three actual conflicts in five years. Simulation-trained AI and combat-tested AI are different things. How different is something you only discover when it matters.

And there are zero ethical debates happening inside Beijing about any of this. The same Georgetown procurement review found nothing resembling the Anthropic-style red lines around autonomous kill chains. A March 2025 paper from PLA-linked researchers described fully autonomous execution of combat decisions in urban environments, including the decision to engage, as a straightforward development goal. Moving that fast toward autonomous lethal AI probably creates real failure modes: systems that misidentify targets, escalate in ways operators can't reverse, behave unpredictably under stress. But the countries that find those limits will be the ones that deployed first.

What Rest of the World Demonstrated

Previously, Ukraine showed the first generation of AI-enabled warfare in practice. AI-assisted drone targeting went from roughly 30-50% accuracy to around 80%. Both sides developed electronic warfare countermeasures and both sides adapted around them. Ukrainian volunteer developers were shipping AI targeting modules for $25 a drone. The whole conflict became a live machine-learning competition where the training data was real battlefield performance.

If Ukraine surprised you, Gaza went further still. Israel deployed a targeting stack with no real precedent in open warfare. The Gospel generated building target lists. Lavender identified individual Hamas members from commanders down to foot soldiers. “Where's Daddy” tracked targets' phones to their homes. The IDF maintained that human validation occurred at the final step, but the pace of operations had compressed that window to seconds.

Iran, this week, is the inverse demonstration. Shahed drones in large numbers. Ballistic missiles aimed at fixed, known targets. The strikes have caused real damage: six American soldiers killed, airports hit across the Gulf, Amazon's data centers offline. But the UAE Ministry of Defense reported intercepting 165 ballistic missiles, two cruise missiles, and 541 Iranian drones since the counterstrikes began. Most of them never arrived. 

When one side has AI-enabled precision and the other is launching at volume without it, that intercept ratio is what the divergence actually looks like in practice.

So Is AI Actually a Competitive Edge?

Yes. Definitively, in 2026. The evidence is running right now over Iranian airspace, and it's been accumulating since 2020.

What it is, specifically, is a significant multiplier on existing military capability. It makes capable militaries faster, more precise, and able to sustain operational tempo that human staff alone could never match. It doesn't transform an underfunded military with bad doctrine into a formidable one.

And the advantage sits on a narrower foundation than it looks. A small number of American companies control the frontier models. Those companies have their own views on what their technology should do, and those views are now demonstrably negotiable under political pressure, in ways that create real instability at the worst possible moments. The operational data that makes battlefield AI good accumulates only through actual conflicts. The talent pipeline for building frontier models doesn't respect borders.

The arms race parallel is real. The Manhattan Project was classified for three years before it changed everything. This race is playing out in corporate press releases, Pentagon procurement notices, and X posts from AI company CEOs, with active strikes in the background and an ongoing negotiation about what the models are even allowed to do.

The window in which the US holds a commanding lead in military AI is open. It is not permanent.

Sources: Al Jazeera, CNBC, Washington Post live conflict coverage (March 2026); Interesting Engineering, "Iran war exposes the expanding role of AI in military strike planning"; MIT Technology Review, "OpenAI's compromise with the Pentagon is what Anthropic feared"; Foreign Affairs, "China's AI Arsenal" (March 2026); CSET, "China's Military AI Wish List" (February 2026); DefenseScoop, GenAI.mil and Pentagon AI coverage; Breaking Defense, "NATO picks Palantir's Maven AI" (April 2025); U.S. Army War College, "AI's Growing Role in Modern Warfare" (August 2025); CSIS, "Technological Evolution on the Battlefield" (October 2025); UK House of Commons Library, "US-Israel strikes on Iran: February/March 2026."

Most enterprises running AI automations at scale are paying for capability they don't use.

They're running invoice extraction, contract parsing, medical claims through frontier model APIs: GPT-4, Claude, Gemini. Processing 10,000 documents daily costs tens of thousands of dollars annually. The accuracy is solid. The latency is acceptable. It works.

Until the vendor ships an update and your accuracy drops. Or your compliance team flags that sensitive data is leaving your infrastructure. Or you realize you're paying for reasoning capabilities you never use to extract the same 12 fields from every invoice.

There's an alternative most teams don't realize is now viable: fine-tuned models purpose-built for your exact document type, deployed on your own infrastructure. Same extraction task. A fraction of the cost. Stable accuracy. Data that never leaves your control.

Let’s decode why.

Why General Models Can Become Unreliable 

When Google launched Gemini 3 in November 2025, the model set new records for reasoning and coding but it removed  pixel-level image segmentation (bounding box masks).

You might think: "We'll just stay on Gemini 2.5 for document extraction." That works until the vendor deprecates the model. OpenAI has deprecated GPT-3, GPT-4-32k, and multiple GPT-4 variants. Anthropic has sunset Claude 2.0 and 2.1. Model lifecycles now run 12-18 months before vendors push migration to newer versions through deprecation notices, pricing changes, or degraded support.

All because the training budget is finite, so when it goes to advanced coding patterns and reasoning chains in general models, it doesn't go to maintaining granular OCR accuracy across edge cases. So when the model is optimized for general capability, specific extraction workflows break.

So the models improve on reasoning, coding, long-context performance but the performance on narrow tasks like structured field extraction, table parsing, and handwritten text recognition changes unpredictably. 

And when you're processing invoices at scale, you need the opposite optimization. Stable, predictable accuracy on a narrow distribution. The invoice schema doesn't change quarter to quarter. The model must extract the same fields with the same accuracy across millions of documents. Frontier models cannot provide this guarantee.

Makes or Breaks at Enterprise Levels

The gap shows up in four places:

Accuracy stability matters more than peak performance. You can't plan around unstable accuracy. A model scoring 94% in January and 91% in March creates operational chaos. Teams built reconciliation workflows assuming 94%. Suddenly 3% more documents need manual review. Batch processing takes longer. Month-end close deadlines slip.

Stable 91% is operationally superior to unstable 94% because you can build reliable processes around known error rates. Frontier model APIs give you no control over when accuracy shifts or in which direction. You're dependent on optimization decisions made for different use cases than yours.

Latency determines throughput capacity. Processing 10,000 invoices per day with 400ms cloud API latency means 66 minutes of pure network overhead before any actual processing. That assumes perfect parallelization and no rate limiting. Real-world API systems hit rate limits, experience variable latency during peak hours, and occasionally face service degradation.

On-premises deployment cuts latency to 50-80ms per document. The same batch completes in 13 minutes instead of 66. This determines whether you can scale to 50,000 documents without infrastructure expansion. API latency creates a ceiling you can't engineer around.

Privacy compliance is binary, not probabilistic. Healthcare claims contain protected health information subject to HIPAA. Financial documents include non-public material information. Legal contracts contain privileged communication.

These cannot transit to vendor infrastructure regardless of encryption, compliance certifications, or contractual terms. Regulatory frameworks and enterprise security policies increasingly require data never leaves controlled environments.

Operational resilience has no API fallback. Manufacturing quality control systems process inspection images in real-time on factory floors. Distribution centers scan shipments continuously regardless of internet availability. Field operations in remote locations have intermittent connectivity.

These workflows require local inference. When network fails, the system continues operating and API-based extraction creates a single point of failure that halts operations. This requires having local fine-tuned models in place.

Where Fine-Tuned Models Actually Win

The difference actually shows up in specific document types where schema complexity and domain knowledge matter more than general intelligence:

Medical billing codes (ICD-10, CPT). The 2026 ICD-10-CM code set contains over 70,000 diagnosis codes. The CPT code set adds 288 new procedure codes. Each diagnosis code must map to appropriate procedure codes based on medical necessity. The relationships are highly structured and domain-specific.

Frontier models struggle because they're optimizing for general medical knowledge, not the specific logic of code pairing and claim validation. Fine-tuned models trained on historical claims data learn the exact patterns insurers accept. AWS documented that fine-tuning on historical clinical data and CMS-1500 form mappings measurably improves code selection precision compared to frontier models.

The complexity: CPT code 99214 (moderate-complexity visit) paired with ICD-10 code E11.9 (Type 2 diabetes) typically processes. The same CPT code paired with Z00.00 (general exam) gets denied. Frontier models lack the training data showing which pairings insurers accept. Fine-tuned models learn this from your claims history.

Legal contract clause extraction. The VLAIR benchmark tested four legal AI tools (Harvey, CoCounsel, Vincent AI, Oliver) and ChatGPT on document extraction tasks. Harvey and CoCounsel, both fine-tuned on legal data: outperformed ChatGPT on clause identification and extraction accuracy.

The difference: legal contracts contain domain-specific terminology and clause structures that follow precedent. "Force majeure," "indemnification," "material adverse change" - these terms have specific legal meanings and typical phrasing patterns. Fine-tuned models trained on contract databases recognize these patterns. Frontier models treat them as general text.

Harvey is built on GPT-4 but fine-tuned specifically on legal corpora. In head-to-head testing, it achieved higher scores on document Q&A and data extraction from contracts than base GPT-4. The improvement comes from training on the specific distribution of legal language and clause structures.

Tax form processing (Schedule C, 1099 variations). Tax forms have highly structured fields with specific validation rules. A Schedule C line 1 (gross receipts) must reconcile with 1099-MISC income reported on line 7. Line 30 (expenses for business use of home) requires Form 8829 attachment if the amount exceeds simplified method limits.

Frontier models don't learn these cross-field validation rules because they're not exposed to sufficient tax form training data during pre-training. Fine-tuned models trained on historical tax returns learn the specific patterns of which fields relate and which combinations trigger validation errors.

Insurance claims with medical necessity documentation. Claims require diagnosis codes justifying the procedure performed. The clinical notes must support the medical necessity. A claim for an MRI (CPT 70553) needs documentation showing why imaging was medically necessary rather than discretionary.

Frontier models evaluate the text as general language. Fine-tuned models trained on approved vs. denied claims learn which documentation patterns insurers accept. The model recognizes that "patient reports persistent headaches unresponsive to medication for 6+ weeks" supports medical necessity for imaging. "Patient requests MRI for peace of mind" does not.

When to Stay on Frontier Models, When to Switch

Most teams choose frontier model APIs because that's what's marketed. But the decision should be well thought.

Keep using frontier models when: The workflow is low-volume, high-stakes reasoning where model capability matters more than cost. Legal contract analysis billed at $400/hour where thoroughness justifies API spend. Strategic research where a single query running for minutes is acceptable. Complex customer support requiring synthesis across multiple systems. Document types vary so significantly that maintaining separate fine-tuned models would be impractical.

These scenarios value capability breadth over cost per inference.

Switch to fine-tuned models deployed on-premises when: The workflow is high-volume, fixed-schema extraction. Invoice processing in AP automation. Medical records parsing for claims. Standard contract review following known templates. Any situation with defined document types, predictable schemas, and volume exceeding 1,000 documents monthly.

The characteristics that justify the switch: accuracy stability over time, latency requirements below 100ms, data that cannot leave your infrastructure, and cost that scales with hardware rather than per-document fees.

The hybrid architecture: Route 90-95% of documents matching standard patterns to fine-tuned models deployed on your infrastructure. These handle known schemas at low cost and high speed. Route the 5-10% of exceptions: unusual formatting, missing fields, ambiguous content to frontier model APIs or human review.

This preserves cost efficiency while maintaining coverage for edge cases. Fine-tuning a lightweight 27B parameter model costs under $10 today. Inference on owned hardware scales with volume at marginal electricity cost. A system processing 10,000 documents daily costs approximately $5k annually for on-premises deployment versus $50k for frontier inference.

Final Thoughts 

Frontier models will keep improving. Benchmark scores will keep rising. The structural mismatch won't change.

General-purpose models optimize for breadth. OpenAI, Anthropic, and Google allocate training budget to whatever drives benchmark scores and API adoption. That's their business model.

Production extraction requires depth. Training budget dedicated to your specific schemas, edge cases, and domain logic. That's your operational requirement.

These targets are incompatible by design. 

And most enterprises default to frontier APIs because that's what's marketed. The tools are polished, the documentation is good, it works well enough to ship. But "works well enough" at tens of thousands annually with unstable accuracy and data leaving your control is different from "works well enough" at a fraction of the cost with stable accuracy on owned infrastructure.

The teams recognizing this early are building systems that will run cheaper and more reliably for years. The teams that don't are paying the frontier model tax on workloads that don't need frontier capabilities.

Which one are you?

Digital Workforce Services Plc invites its investors and analysts to an Investor Day on Thursday March 19, 2026 at 14-16 EET. Preliminary agenda of the day:

CEO Jussi Vasama will outline the company’s strategic priorities and the key 2026 objectives for its new business areas.

CFO Laura Viita will walk through the company’s financial performance and targets.

Karli Kalpala, Head of Strategy and AI Business, will present the company’s AI strategy, AI agent–driven product portfolio, and related partnerships.

Juha Nieminen, Chief Growth Officer of Healthcare business area, will discuss the healthcare automation market, growth outlook, and recent customer implementations.

The event takes place in Flik Studio Eliel, Sanoma House (address: Töölönlahdenkatu 2), and coffee will be served to participants before the program begins.

Participants attending on-site are kindly asked to register by Tuesday, 17 March 2026 via email to address finance@digitalworkforce.com.

The event will be held in English.

In addition to the on-site event, the session will be streamed live as a webcast starting at 14:00 EET. Participants will have the opportunity to submit questions to the speakers via the webcast platform’s chat function. The webcast link will be published on the company’s website prior to the event.

All presentation materials, as well as a recording of the event, will be published on the company’s website Reports and presentations | Digital Workforce.

We warmly welcome you to join the Digital Workforce Investor Day!

Contact information:

Digital Workforce Services Plc

Jussi Vasama, CEO
Tel. +358 50 380 9893

Laura Viita, CFO
Tel. +358 50 487 1044

Investor relations | Digital Workforce

Press release 5.3.2026, 8:00 EET: Digital Workforce Services Plc hosts an Investor Day on March 19, 2026 at 14-16 EET

The post Digital Workforce Services Plc hosts an Investor Day on March 19, 2026 at 14-16 EET appeared first on Digital Workforce.

On February 13, the Wall Street Journal reported something that hadn't been public before: the Pentagon used Anthropic's Claude AI during the January raid that captured Venezuelan Leader Nicolás Maduro.

It said Claude's deployment came through Anthropic's partnership with Palantir Technologies, whose platforms are widely used by the Defense Department.

Reuters attempted to independently verify the report - they couldn't. Anthropic declined to comment on specific operations. The Department of Defense declined to comment. Palantir said nothing.

But the WSJ report revealed one more detail.

Sometime after the January raid, an Anthropic employee reached out to someone at Palantir and asked a direct question: how was Claude actually used in that operation?

The company that built the model and signed the $200 million contract had to ask someone else what their own software did during a military attack on a capital city.

This one detail tells you everything about where we actually are with AI governance. It also tells you why "human in the loop" stopped being a safety guarantee somewhere between the contract signing and Caracas.

How big was the operation

Calling this a covert extraction misses what actually happened.

Delta Force raided multiple targets across Caracas. More than 150 aircraft were involved. Air defense systems were suppressed before the first boots hit the ground. Airstrikes hit military targets and air defenses, and electronic warfare assets were moved into the region, per Reuters.

Cuba later confirmed 32 of its soldiers and intelligence personnel were killed and declared two days of national mourning. Venezuela's government cited a death toll of roughly 100.

Two sources told Axios that Claude was used during the active operation itself, though Axios noted it could not confirm the precise role Claude played.

What Claude might actually have done 

To understand what could have been happening, you need to know one technical thing about how Claude works.

Anthropic's API is stateless. Each call is independent i.e. you send text in, you get text back, and that interaction is over. There's no persistent memory or Claude running continuously in the background.

It's less like a brain and more like an extremely fast consultant you can call every thirty seconds: you describe the situation, they give you their best analysis, you hang up, you call again with new information.

That's the API. But that says nothing about the systems Palantir built on top of it.

You can engineer an agent loop that feeds real-time intelligence into Claude continuously. You can build workflows where Claude's outputs trigger the next action with minimal latency between recommendation and execution.

Testing These Scenarios Myself

To understand what this actually looks like in practice, I tested some of these scenarios.

every 30 seconds. indefinitely.

The API is stateless. A sophisticated military system built on the API doesn't have to be.

What that might look like when deployed: 

Intercepted communications in Spanish fed to Claude for instant translation and pattern analysis across hundreds of messages simultaneously. Satellite imagery processed to identify vehicle movements, troop positions, or infrastructure changes with updates every few minutes as new images arrived. 

Or real-time synthesis of intelligence from multiple sources - signals intercepts, human intelligence reports, electronic warfare data - compressed into actionable briefings that would take analysts hours to produce manually.

 trained on scenarios. deployed in Caracas.

None of that requires Claude to "decide" anything. It's all analysis and synthesis.

But when you're compressing a four-hour intelligence cycle into minutes, and that analysis is feeding directly into operational decisions being made at that same compressed timescale, the distinction between "analysis" and "decision-making" starts to collapse.

And because this is a classified network, nobody outside that system knows what was actually built.

So when someone says "Claude can't run an autonomous operation" - they're probably right about the API level. Whether they're right about the deployment level is a completely different question. And one nobody can currently answer.

Gap between autonomous and meaningful

Anthropic's hard limit is autonomous weapons - systems that decide to kill without a human signing off. That's a real line.

But there's an enormous amount of territory between "autonomous weapons" and "meaningful human oversight." Think about what it means in practice for a commander in an active operation. Claude is synthesizing intelligence across data volumes no analyst could hold in their head. It's compressing what used to be a four-hour briefing cycle into minutes.

this took 3 seconds.

It's surfacing patterns and recommendations faster than any human team could produce them.

Technically, a human approves everything before any action is taken. The human is in the process. But the process is now moving so fast that it becomes impossible to evaluate what’s in it in fast paced scenarios like a military attack.When Claude generates an intelligence summary, that summary becomes the input for the next decision. And because Claude can produce these summaries so much faster than humans can process them, the pace of the entire operation speeds up.

You can't slow down to think carefully about a recommendation when the situation it describes is already three minutes old. The information has moved on. The next update is already arriving. The loop keeps getting faster.

90 seconds to decide. this is what the loop looks like from inside.

The requirement for human approval is there but the ability to meaningfully evaluate what you're approving is not.

And it gets structurally worse the better the AI gets because better AI means faster synthesis, shorter decision windows, less time to think before acting.

Pentagon and Claude’s arguments

The Pentagon wants access to AI models for any use case that complies with U.S. law. Their position is essentially: usage policy is our problem, not yours.

But Anthropic wants to maintain specific prohibitions - no fully autonomous weapons and prohibiting mass domestic surveillance of Americans.

After the WSJ broke the story, a senior administration official told Axios their partnership/agreement was under review and this is the reason Pentagon stated:

"Any company that would jeopardize the operational success of our warfighters in the field is one we need to reevaluate."

But ironically, Anthropic is currently the only commercial AI model approved for certain classified DoD networks. Although, OpenAI, Google, and xAI are all actively in discussions to get onto those systems with fewer restrictions.

The real fight beyond arguments

In hindsight, Anthropic and the Pentagon might be missing the entire point and thinking policy languages might solve this issue.

Contracts can mandate human approval at every step. But, that does not mean the human has enough time, context, or cognitive bandwidth to actually evaluate what they're approving. That gap between a human technically in the loop and a human actually able to think clearly about what's in it is where the real risk lives.

Rogue AI and autonomous weapons are probably the later set of arguments.

Today’s debate should be - would you call it “supervised” when you put a system that processes information orders of magnitude faster than humans into a human command chain?

Final thoughts

In Caracas, in January, with 150 aircraft and real-time feeds and decisions being made at operational speed and we don't know the answer to that.

And neither does Anthropic.

But soon, with fewer restrictions in place and more models on those classified networks, we're all going to find out.

All claims in this piece are sourced to public reporting and documented specifications. We have no non-public information about this operation. Sources: WSJ (Feb 13), Axios (Feb 13, Feb 15), Reuters (Jan 3, Feb 13). Casualty figures from Cuba's official government statement and Venezuela's defense ministry. API architecture from platform.claude.com/docs. Contract details from Anthropic's August 2025 press release. "Visibility into usage" quote from Axios (Feb 13).

Intro: The Story We All Know

You build an AI agent on Friday afternoon. You demo it to your team Monday morning. The agent qualifies leads smoothly, books meetings without asking twice, and even generates proposals on the fly. Your manager nods approvingly.

Two weeks later, it's in production. What could go wrong? ?

By Wednesday, customers are complaining: "Why does the bot keep asking me my company name when I already told it?" By Friday, you're debugging why the bot booked a meeting for the wrong date. By the following Monday, you've silently rolled it back.

What went wrong? Model is the same in demo and prod. It was something much more fundamental: your agent can't reliably pass and manage variables across steps. Your agent also lacks proper identity controls to prevent accessing variables it shouldn't.

What Is a Variable (And Why It Matters)

A variable is just a named piece of information your agent needs to remember or use:

• Customer name
• Order ID
• Selected product
• Meeting date
• Task progress
• API response

Variable passing is how that information flows from one step to the next without getting lost or corrupted.

Think of it like filling a multi-page form. Page 1: you enter your name and email. Page 2: the form should already show your name and email, not ask again. If the system doesn't "pass" those fields from Page 1 to Page 2, the form feels broken. That's exactly what's happening with your agent.

Why This Matters in Production

LLMs are fundamentally stateless. A language model is like a person with severe amnesia. Every time you ask it a question, it has zero memory of what you said before unless you explicitly remind it by including that information in the prompt.

(Yes, your agent has the memory of a goldfish. No offense to goldfish. ?)

If your agent doesn't explicitly store and pass user data, context, and tool outputs from one step to the next, the agent literally forgets everything and has to start over.

In a 2-turn conversation? Fine, the context window still has room. In a 10-turn conversation where the agent needs to remember a customer's preferences, previous decisions, and API responses? The context window fills up, gets truncated, and your agent "forgets" critical information.

This is why it works in demo (short conversations) but fails in production (longer workflows).

The Four Pain Points

Pain Point 1: The Forgetful Assistant

After 3-4 conversation turns, the agent forgets user inputs and keeps asking the same questions repeatedly.

Why it happens:

• Relying purely on prompt context (which has limits)
• No explicit state storage mechanism
• Context window gets bloated and truncated

Real-world impact:

User: "My name is Priya and I work at TechCorp"
Agent: "Got it, Priya at TechCorp. What's your biggest challenge?"
User: "Scaling our infrastructure costs"
Agent: "Thanks for sharing. Just to confirm—what's your name and company?"
User: ?

At this point, Priya is questioning whether AI will actually take her job or if she'll die of old age before the agent remembers her name.

Pain Point 2: Scope Confusion Problem

Variables defined in prompts don't match runtime expectations. Tool calls fail because parameters are missing or misnamed.

Why it happens:

• Mismatch between what the prompt defines and what tools expect
• Fragmented variable definitions scattered across prompts, code, and tool specs

Real-world impact:

Prompt says: "Use customer_id to fetch the order"
Tool expects: "customer_uid"
Agent tries: "customer_id"
Tool fails

Pain Point 3: UUIDs Get Mangled

LLMs are pattern matchers, not randomness engines. A UUID is deliberately high-entropy, so the model often produces something that looks like a UUID (right length, hyphens) but contains subtle typos, truncations, or swapped characters. In long chains, this becomes a silent killer: one wrong character and your API call is now targeting a different object, or nothing at all.

If you want a concrete benchmark, Boundary’s write-up shows a big jump in identifier errors when prompts contain direct UUIDs, and how remapping to small integers significantly improves accuracy (UUID swap experiment).

How teams avoid this: don’t ask the model to handle UUIDs directly. Use short IDs in the prompt (001, 002 or ITEM-1, ITEM-2), enforce enum constraints where possible, and map back to UUIDs in code. (You’ll see these patterns again in the workaround section below.)

Pain Point 4: Chaotic Handoffs in Multi-Agent Systems

Data is passed as unstructured text instead of structured payloads. Next agent misinterprets context or loses fidelity.

Why it happens:

• Passing entire conversation history instead of structured state
• No clear contract for inter-agent communication

Real-world impact:

Agent A concludes: "Customer is interested"
Passes to Agent B as: "Customer says they might be interested in learning more"
Agent B interprets: "Not interested yet"
Agent B decides: "Don't book a meeting"
→ Contradiction.

Pain Point 5: Agentic Identity (Concurrency & Corruption)

Multiple users or parallel agent runs race on shared variables. State gets corrupted or mixed between sessions.

Why it happens:

• No session isolation or user-scoped state
• Treating agents as stateless functions
• No agentic identity controls

Real-world impact (2024):

User A's lead data gets mixed with User B's lead data.
User A sees User B's meeting booked in their calendar.
→ GDPR violation. Lawsuit incoming.

Your legal team's reaction: ???

Real-world impact (2026):

Lead Scorer Agent reads Salesforce
It has access to Customer ID = cust_123
But which customer_id? The one for User A or User B?

Without agentic identity, it might pull the wrong customer data
→ Agent processes wrong data
→ Wrong recommendations

The 2026 Memory Stack: Episodic, Semantic, and Procedural

Modern agents now use Long-Term Memory Modules (like Google's Titans architecture and test-time memorization) that can handle context windows larger than 2 million tokens by incorporating "surprise" metrics to decide what to remember in real-time.

But even with these advances, you still need explicit state management. Why?

1. Memory without identity control means an agent might access customer data it shouldn't
2. Replay requires traces: long-term memory helps, but you still need episodic traces (exact logs) for debugging and compliance
3. Speed matters: even with 2M token windows, fetching from a database is faster than scanning through 2M tokens

By 2026, the industry has moved beyond "just use a database" to Memory as a first-class design primitive. When you design variable passing now, think about three types of memory your agent needs to manage:

1. Episodic Memory (What happened in this session)

The action traces and exact events that occurred. Perfect for replay and debugging.

{
  "session_id": "sess_123",
  "timestamp": "2026-02-03 14:05:12",
  "action": "check_budget",
  "tool": "salesforce_api",
  "input": { "customer_id": "cust_123" },
  "output": { "budget": 50000 },
  "agent_id": "lead_scorer_v2"
}

Why it matters:

• Replay exact sequence of events
• Debug "why did the agent do that?"
• Compliance audits
• Learn from failures

2. Semantic Memory (What the agent knows)

Think of this as your agent's "wisdom from experience." The patterns it learns over time without retraining. For example, your lead scorer learns: SaaS companies close at 62% (when qualified), enterprise deals take 4 weeks on average, ops leaders decide in 2 weeks while CFOs take 4.

This knowledge compounds across sessions. The agent gets smarter without you lifting a finger.

{
  "agent_id": "lead_scorer_v2",
  "learned_patterns": {
    "conversion_rates": {
      "saas_companies": 0.62,
      "enterprise": 0.58,
      "startups": 0.45
    },
    "decision_timelines": {
      "ops_leaders": "2 weeks",
      "cfo": "4 weeks",
      "cto": "3 weeks"
    }
  },
  "last_updated": "2026-02-01",
  "confidence": 0.92
}

Why it matters: agents learn from experience, better decisions over time, cross-session learning without retraining. Your lead scorer gets 15% more accurate over 3 months without touching the model.

3. Procedural Memory (How the agent operates)

The recipes or standard operating procedures the agent follows. Ensures consistency.

{
  "workflow_id": "lead_qualification_v2.1",
  "version": "2.1",
  "steps": [
    {
      "step": 1,
      "name": "collect",
      "required_fields": ["name", "company", "budget"],
      "description": "Gather lead basics"
    },
    {
      "step": 2,
      "name": "qualify",
      "scoring_criteria": "check fit, timeline, budget",
      "min_score": 75
    },
    {
      "step": 3,
      "name": "book",
      "conditions": "score >= 75",
      "actions": ["check_calendar", "book_meeting"]
    }
  ]
}

Why it matters: standard operating procedures ensure consistency, easy to update workflows (version control), new team members understand agent behavior, easier to debug ("which step failed?").

The Protocol Moment: "HTTP for AI Agents"

In late 2025, the AI agent world had a problem: every tool worked differently, every integration was custom, and debugging was a nightmare. A few standards and proposals started showing up, but the practical fix is simpler: treat tools like APIs, and make every call schema-first.

Think of tool calling (sometimes called function calling) like HTTP for agents. Give the model a clear, typed contract for each tool, and suddenly variables stop leaking across steps.

The Problem Protocols (and Tool Calling) Solve

Without schemas (2024 chaos):

Agent says: "Call the calendar API"
Calendar tool responds: "I need customer_id and format it as UUID"
Agent tries: { "customer_id": "123" }
Tool says: "That's not a valid UUID"
Agent retries: { "customer_uid": "cust-123-abc" }
Tool says: "Wrong field name, I need customer_id"
Agent: ?

(This is Pain Point 2: Scope Confusion)

With schema-first tool calling, your tool layer publishes a tool catalog:

{
  "tools": [
    {
      "name": "check_calendar",
      "input_schema": {
        "customer_id": { "type": "string", "format": "uuid" }
      },
      "output_schema": {
        "available_slots": [{ "type": "datetime" }]
      }
    }
  ]
}

Agent reads catalog once. Agent knows exactly what to pass. Agent constructs { "customer_id": "550e8400-e29b-41d4-a716-446655440000" }. Tool validates using schema. Tool responds { "available_slots": [...] }. ✅ Zero confusion, no retries and hallucination.

Real-World 2026 Status

Most production stacks are converging on the same idea: schema-first tool calling. Some ecosystems wrap it in protocols, some ship adapters, and some keep it simple with JSON schema tool definitions.

LangGraph (popular in 2026): a clean way to make variable flow explicit via a state machine, while still using the same tool contracts underneath.

Net takeaway: connectors and protocols will be in flux (Google’s UCP is a recent example in commerce), but tool calling is the stable primitive you can design around.

Impact on Pain Point 2: Scope Confusion is Solved

By adopting schema-first tool calling, variable names match exactly (schema enforced), type mismatches are caught before tool calls, and output formats stay predictable. No more "does the tool expect customer_id or customer_uid?"

2026 Status: LARGELY SOLVED ✅. Schema-first tool calling means variable names and types are validated against contracts early. Most teams don't see this anymore once they stop hand-rolling integrations.

2026 Solution: Agentic Identity Management

By 2026, best practice is to use OAuth 2.1 profiles specifically for agents.

{
  "agent_id": "lead_scorer_v2",
  "oauth_token": "agent_token_xyz",
  "permissions": {
    "salesforce": "read:leads,accounts",
    "hubspot": "read:contacts",
    "calendar": "read:availability"
  },
  "user_scoped": {
    "user_id": "user_123",
    "tenant_id": "org_456"
  }
}

When Agent accesses a variable: Agent says "Get customer data for customer_id = 123". Identity system checks "Agent has permissions? YES". Identity system checks "Is customer_id in user_123's tenant? YES". System provides customer data. ✅ No data leakage between tenants.

The Four Methods to Pass Variables

Method 1: Direct Pass (The Simple One)

Variables pass immediately from one step to the next.

Step 1 computes: total_amount = 5000
       ↓
Step 2 immediately receives total_amount
       ↓
Step 3 uses total_amount

Best for: simple, linear workflows (2-3 steps max), one-off tasks, speed-critical applications.

2026 Enhancement: add schema/type validation even for direct passes (tool calling). Catches bugs early.

from pydantic import BaseModel

class TotalOut(BaseModel):
    total_amount: float

def calculate_total(items: list[dict]) -> dict:
    total = sum(item["price"] for item in items)
    return TotalOut(total_amount=total).model_dump()

Method 2: Variable Repository (The Reliable One)

Shared storage (database, Redis) where all steps read/write variables.

Step 1 stores: customer_name, order_id
       ↓
Step 5 reads: same values (no re-asking)

2026 Architecture (with Memory Types):

# Episodic Memory: Exact action traces
episodic_store = {
  "session_id": "sess_123",
  "traces": [
    {
      "timestamp": "2026-02-03 14:05:12",
      "action": "asked_for_budget",
      "result": "$50k",
      "agent": "lead_scorer_v2"
    }
  ]
}

# Semantic Memory: Learned patterns
semantic_store = {
  "agent_id": "lead_scorer_v2",
  "learned": {
    "saas_to_close_rate": 0.62
  }
}

# Procedural Memory: Workflows
procedural_store = {
  "workflow_id": "lead_qualification",
  "steps": [...]
}

# Identity layer (NEW 2026)
identity_layer = {
  "agent_id": "lead_scorer_v2",
  "user_id": "user_123",
  "permissions": "read:leads, write:qualification_score"
}

Who uses this (2026): yellow.ai, Agent.ai, Amazon Bedrock Agents, CrewAI (with tool calling + identity layer).

Best for: multi-step workflows (3+ steps), multi-turn conversations, production systems with concurrent users.

Method 3: File System (The Debugger's Best Friend)

Variables saved as files (JSON, logs). Still excellent for code generation and sandboxed agents (Manus, AgentFS, Dust).

Best for: long-running tasks, code generation agents, when you need perfect audit trails.

Method 4: State Machines + Database (The Gold Standard)

Explicit state machine with database persistence. Transitions are code-enforced. 2026 Update: "Checkpoint-Aware" State Machines.

state_machine = {
  "current_state": "qualification",
  "checkpoint": {
    "timestamp": "2026-02-03 14:05:26",
    "state_data": {...},
    "recovery_point": True  # ← If agent crashes here, it resumes from checkpoint
  }
}

Real companies using this (2026): LangGraph (graph-driven, checkpoint-aware), CrewAI (role-based, with tool calling + state machine), AutoGen (conversation-centric, with recovery), Temporal (enterprise workflows).

Best for: complex, multi-step agents (5+ steps), production systems at scale, mission-critical, regulated environments.

The 2026 Framework Comparison

How to Pick the Best Method: Updated Decision Framework

? Quick Decision Flowchart

By Agent Complexity

By Use Case (2026)

Real Example: How to Pick

Scenario: Lead qualification agent

Requirements: (1) Collect lead info (name, company, budget), (2) Ask qualifying questions, (3) Score the lead, (4) Book a meeting if qualified, (5) Send follow-up email.

Decision Process (2026):

Q1: How many steps? A: 5 steps → Not Direct Pass ❌

Q2: Does it need to survive failures? A: Yes, can't lose lead data → Need State Machine ✅

Q3: Multiple agents involved? A: Yes (scorer + booker + email sender) → Need tool calling ✅

Q4: Multi-tenant (multiple users)? A: Yes → Need Agentic Identity ✅

Q5: How mission-critical? A: Drives revenue → Need audit trail ✅

Q6: Engineering capacity? A: Small team, ship fast → Use LangGraph ✅

(LangGraph handles state machine + tool calling + checkpoints)

2026 Architecture:

from typing import TypedDict
from langgraph.graph import StateGraph, START, END
from langgraph.checkpoint.memory import MemorySaver

# Define state structure
class AgentState(TypedDict):
    # Lead data
    customer_name: str
    company: str
    budget: int
    score: int
    
    # Identity context (passed through state)
    user_id: str
    tenant_id: str
    oauth_token: str
    
    # Memory references
    episodic_trace: list
    learned_patterns: dict

# Create graph with state
workflow = StateGraph(AgentState)

# Add nodes
workflow.add_node("collect", collect_lead_info)
workflow.add_node("qualify", ask_qualifying_questions)
workflow.add_node("score", score_lead)
workflow.add_node("book", book_if_qualified)
workflow.add_node("followup", send_followup_email)

# Define edges
workflow.add_edge(START, "collect")
workflow.add_edge("collect", "qualify")
workflow.add_edge("qualify", "score")
workflow.add_conditional_edges(
    "score",
    lambda state: "book" if state["score"] >= 75 else "followup"
)
workflow.add_edge("book", "followup")
workflow.add_edge("followup", END)

# Compile with checkpoints (CRITICAL: Don't forget this!)
checkpointer = MemorySaver()
app = workflow.compile(checkpointer=checkpointer)

# tool-calling-ready tools
tools = [
    check_calendar,  # tool-calling-ready
    book_meeting,    # tool-calling-ready
    send_email       # tool-calling-ready
]

# Run with identity in initial state
initial_state = {
    "user_id": "user_123",
    "tenant_id": "org_456",
    "oauth_token": "agent_oauth_xyz",
    "episodic_trace": [],
    "learned_patterns": {}
}

# Execute with checkpoint recovery enabled
result = app.invoke(
    initial_state,
    config={"configurable": {"thread_id": "sess_123"}}
)

app = workflow.compile()

from langgraph.checkpoint.memory import MemorySaver
app = workflow.compile(checkpointer=MemorySaver())

Result: state machine enforces "collect → qualify → score → book → followup", agentic identity prevents accessing wrong customer data, episodic memory logs every action (replay for debugging), tool calling ensures tools are called with correct parameters, checkpoints allow recovery if agent crashes, full audit trail for compliance.

Best Practices for 2026

1. ? Define Your Memory Stack

Your memory architecture determines how well your agent learns and recovers. Choose stores that match each memory type's purpose: fast databases for episodic traces, vector databases for semantic patterns, and version control for procedural workflows.

{
  "episodic": {
    "store": "PostgreSQL",
    "retention": "90 days",
    "purpose": "Replay and debugging"
  },
  "semantic": {
    "store": "Vector DB (Pinecone/Weaviate)",
    "retention": "Indefinite",
    "purpose": "Cross-session learning"
  },
  "procedural": {
    "store": "Git + Config Server",
    "retention": "Versioned",
    "purpose": "Workflow definitions"
  }
}

This setup gives you replay capabilities (PostgreSQL), cross-session learning (Pinecone), and workflow versioning (Git). Production teams report 40% faster debugging with proper memory separation.

Practical Implementation:

# 1. Episodic Memory (PostgreSQL)
from sqlalchemy import create_engine, Column, String, JSON, DateTime
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

Base = declarative_base()

class EpisodicTrace(Base):
    __tablename__ = 'episodic_traces'
    
    id = Column(String, primary_key=True)
    session_id = Column(String, index=True)
    timestamp = Column(DateTime, index=True)
    action = Column(String)
    tool = Column(String)
    input_data = Column(JSON)
    output_data = Column(JSON)
    agent_id = Column(String, index=True)
    user_id = Column(String, index=True)

engine = create_engine('postgresql://localhost/agent_memory')
Base.metadata.create_all(engine)

# 2. Semantic Memory (Vector DB)
from pinecone import Pinecone

pc = Pinecone(api_key="your-api-key")
semantic_index = pc.Index("agent-learnings")

# Store learned patterns
semantic_index.upsert(vectors=[{
    "id": "lead_scorer_v2_pattern_1",
    "values": embedding,  # Vector embedding of the pattern
    "metadata": {
        "agent_id": "lead_scorer_v2",
        "pattern_type": "conversion_rate",
        "industry": "saas",
        "value": 0.62,
        "confidence": 0.92
    }
}])

# 3. Procedural Memory (Git + Config Server)
import yaml

workflow_definition = {
    "workflow_id": "lead_qualification",
    "version": "2.1",
    "changelog": "Added budget verification",
    "steps": [
        {"step": 1, "name": "collect", "required_fields": ["name", "company", "budget"]},
        {"step": 2, "name": "qualify", "scoring_criteria": "fit, timeline, budget"},
        {"step": 3, "name": "book", "conditions": "score >= 75"}
    ]
}

with open('workflows/lead_qualification_v2.1.yaml', 'w') as f:
    yaml.dump(workflow_definition, f)

2. ? Adopt Tool Calling From Day One

Tool calling eliminates variable naming mismatches and makes tools self-documenting. Instead of maintaining separate API docs, your tool definitions include schemas that agents can read and validate against automatically.

Every tool should be schema-first so agents can auto-discover and validate them.

# Tool calling (function calling) = schema-first contracts for tools

tools = [
  {
    "type": "function",
    "function": {
      "name": "check_calendar",
      "description": "Check calendar availability for a customer",
      "parameters": {
        "type": "object",
        "properties": {
          "customer_id": {"type": "string"},
          "start_date": {"type": "string"},
          "end_date": {"type": "string"}
        },
        "required": ["customer_id", "start_date", "end_date"]
      }
    }
  }
]

# Your agent passes this tool schema to the model.
# The model returns a structured tool call with args that match the contract.

Now agents can auto-discover and validate this tool without manual integration work.

3. ? Implement Agentic Identity (OAuth 2.1 for Agents)

Just as users need permissions, agents need scoped access to data. Without identity controls, a lead scorer might accidentally access customer data from the wrong tenant, creating security violations and compliance issues.

2026 approach: Agents have OAuth tokens, just like users do.

# Define agent context with OAuth 2.1
agent_context = {
    "agent_id": "lead_scorer_v2",
    "user_id": "user_123",
    "tenant_id": "org_456",
    "oauth_token": "agent_token_xyz",
    "scopes": ["read:leads", "write:qualification_score"]
}

When agent accesses a variable, identity is checked:

from functools import wraps
from typing import Callable, Any
from datetime import datetime

class PermissionError(Exception):
    pass

class SecurityError(Exception):
    pass

def check_agent_permissions(func: Callable) -> Callable:
    """Decorator to enforce identity checks on variable access"""
    @wraps(func)
    def wrapper(var_name: str, agent_context: dict, *args, **kwargs) -> Any:
        # 1. Check if agent has permission to access this variable type
        required_scope = get_required_scope(var_name)
        if required_scope not in agent_context.get('scopes', []):
            raise PermissionError(
                f"Agent {agent_context['agent_id']} lacks scope '{required_scope}' "
                f"required to access {var_name}"
            )
        
        # 2. Check if variable belongs to agent's tenant
        variable_tenant = get_variable_tenant(var_name)
        agent_tenant = agent_context.get('tenant_id')
        
        if variable_tenant != agent_tenant:
            raise SecurityError(
                f"Variable {var_name} belongs to tenant {variable_tenant}, "
                f"but agent is in tenant {agent_tenant}"
            )
        
        # 3. Log the access for audit trail
        log_variable_access(
            agent_id=agent_context['agent_id'],
            user_id=agent_context['user_id'],
            variable_name=var_name,
            access_type='read',
            timestamp=datetime.utcnow()
        )
        
        return func(var_name, agent_context, *args, **kwargs)
    
    return wrapper

def get_required_scope(var_name: str) -> str:
    """Map variable names to required OAuth scopes"""
    scope_mapping = {
        'customer_name': 'read:leads',
        'customer_email': 'read:leads',
        'customer_budget': 'read:leads',
        'qualification_score': 'write:qualification_score',
        'meeting_scheduled': 'write:calendar'
    }
    return scope_mapping.get(var_name, 'read:basic')

def get_variable_tenant(var_name: str) -> str:
    """Retrieve the tenant ID associated with a variable"""
    # In production, this would query your variable repository
    from database import variable_store
    variable = variable_store.get(var_name)
    return variable['tenant_id'] if variable else None

def log_variable_access(agent_id: str, user_id: str, variable_name: str, 
                       access_type: str, timestamp: datetime) -> None:
    """Log all variable access for compliance and debugging"""
    from database import audit_log
    audit_log.insert({
        'agent_id': agent_id,
        'user_id': user_id,
        'variable_name': variable_name,
        'access_type': access_type,
        'timestamp': timestamp
    })

@check_agent_permissions
def access_variable(var_name: str, agent_context: dict) -> Any:
    """Fetch variable with identity checks"""
    from database import variable_store
    return variable_store.get(var_name)

# Usage
try:
    customer_budget = access_variable('customer_budget', agent_context)
except PermissionError as e:
    print(f"Access denied: {e}")
except SecurityError as e:
    print(f"Security violation: {e}")

This decorator pattern ensures every variable access is logged, scoped, and auditable. Multi-tenant SaaS platforms using this approach report zero cross-tenant data leaks.

4. ⚙️ Make State Machines Checkpoint-Aware

Checkpoints let your agent resume from failure points instead of restarting from scratch. This saves tokens, reduces latency, and prevents data loss when crashes happen mid-workflow.

2026 pattern: Automatic recovery

# Add checkpoints after critical steps
state_machine.add_checkpoint_after_step("collect")
state_machine.add_checkpoint_after_step("qualify")
state_machine.add_checkpoint_after_step("score")

# If agent crashes at "book", restart from "score" checkpoint
# Not from beginning (saves time and money)

In production, this means a 30-second workflow doesn't need to repeat the first 25 seconds just because the final step failed. LangGraph and Temporal both support this natively.

5. ? Version Everything (Including Workflows)

Treat workflows like code: deploy v2.1 alongside v2.0, roll back easily if issues arise.

# Version your workflows
workflow_v2_1 = {
    "version": "2.1",
    "changelog": "Added budget verification before booking",
    "steps": [...]
}

Versioning lets you A/B test workflow changes, roll back bad deploys instantly, and maintain audit trails for compliance. Store workflows in Git alongside your code for single-source-of-truth version control.

6. ? Build Observability In From Day One

Without observability, debugging a multi-step agent is guesswork. With it, you can replay exact sequences, identify bottlenecks, and prove compliance. Teams with proper observability resolve production issues 3x faster.

The 2026 Architecture Stack

Here's what a production agent looks like in 2026:

Your 2026 Checklist: Before You Ship

Before deploying your agent to production, verify:

Conclusion: The 2026 Agentic Future

The agents that win in 2026 will need more than just better prompts. They're the ones with proper state management, schema-standardized tool access, agentic identity controls, three-tier memory architecture, checkpoint-aware recovery and full observability.

State Management and Identity and Access Control are probably the hardest parts about building AI agents.

Now you know how to get both right.

Last Updated: February 3, 2026

Start building. ?

About This Guide

This guide was written in February 2026, reflecting the current state of AI agent development. It incorporates lessons learned from production deployments at Nanonets Agents and also from the best practices we noticed in the current ecosystem.

Version: 2.1
Last Updated: February 3, 2026

Deal includes access to SS&C Blue Prism automations

Digital Workforce, a global leader in enterprise automation and AI-driven solutions, is proud to announce a landmark deal with one of the largest integrated academic health systems in the world. The U.S. based client organization employs over 80 000 people and comprises world-leading hospitals and a vast research enterprise. The newly signed partnership marks a significant milestone in the health system’s journey to future-proof its automation capabilities and scale intelligent operations across its organization. The yearly value of the agreement is 1,4M USD.

Under the new contract, Digital Workforce will support the client in modernizing and migrating its substantial on-premise automation infrastructure with over 100 bots to a secure, scalable cloud environment. The transition includes deploying the Digital Workforce Outsmart cloud platform to provide flexible, consumption-based access to SS&C Blue Prism technology, supported by 24/7 managed services from Digital Workforce. The collaboration also opens new opportunities for the client to expand automation across clinical and administrative pathways in the future, such as intelligent document processing, AI agent integration, and enterprise-wide automation governance.

“This deal exemplifies our commitment to delivering measurable value to large healthcare organizations through enterprise-wide automation, enabling the fast and secure deployment of solutions that improve the reliability, efficiency, and safety of critical processes,” said Karri Lehtonen, Head of Digital Workforce North America: “By combining multi-technology offering and cloud flexibility with robust managed services, we’ve laid the foundation for long-term innovation and operational excellence.”

“Our close partnership with Digital Workforce spans more than a decade. The company’s expertise in process excellence and service delivery is exceptional, particularly in the healthcare sector, where our companies have long shared a strong focus. We are proud to see our technology supporting this world-leading healthcare system and to collaborate with Digital Workforce in transforming one of the most critical industries through agentic automation,” said Rob Stone, General Manager, IA & Analytics at SS&C Technologies.

“This latest win underscores Digital Workforce’s position as a trusted partner for large-scale automation transformation programs in regulated industries, like healthcare, where quality, compliance, and innovation must go hand in hand. Moreover, we are exceedingly proud to support this customer specifically: a world-renowned, research-intensive healthcare system and one of the largest in the United States. To put that into a European perspective, the operating revenue of the healthcare system is close to Finland’s total public expenditure on healthcare,” described Jussi Vasama, Digital Workforce CEO.

 

For media enquiries please contact:

jamie.dootson@sscinc.com

About Digital Workforce Services Plc

Digital Workforce Services Plc (Nasdaq First North: DWF) is a leader in business automation and technology solutions. With the Digital Workforce Outsmart platform and services—including Enterprise AI agents—organizations transform knowledge work, reduce costs, accelerate digitization, grow revenue, and improve customer experience. More than 200 large customers use our services to drive the transformation of work through automation and Agentic AI. Digital Workforce has particularly strong experience in healthcare, automating care pathways across clinical and administrative workflows to reduce burden, enhance patient safety, and return time to patient care. Following the acquisition of e18 Innovation, the company has further strengthened its position in the UK healthcare pathway automation. We focus on repeatable, outcome-based use cases, and we operate with high integrity and close customer collaboration. Founded in 2015, Digital Workforce employs more than 200 automation professionals in the US, UK, Ireland, and Northern and Central Europe. Our vision: Transforming Work – Beyond Productivity. https://digitalworkforce.com

About SS&C Technologies

SS&C is a global provider of services and software for the financial services and healthcare industries. Founded in 1986, SS&C is headquartered in Windsor, Connecticut, and has offices around the world. More than 23,000 financial services and healthcare organizations, from the world’s largest companies to small and mid-market firms, rely on SS&C for expertise, scale and technology.

Press release: Digital Workforce Secures Deal Annually Valued at 1,4M $ with U.S. Academic Health System – Customer Comparable in Scale to European National Health Services

The post Digital Workforce Secures Deal Annually Valued at 1,4M $ with U.S. Academic Health System – Customer Comparable in Scale to European National Health Services appeared first on Digital Workforce.

Digital Workforce today announced the successful production deployment of an enterprise AI Agent with a leading European property and casualty insurer. The AI Agent automates key parts of personal injury claims processing and has moved from a rigorous production pilot into live operations, showing how agentic AI can be adopted safely in complex, regulated environments.

Faster, more consistent service provider optimisation, without removing human control
The AI Agent supports personal injury claims handling by optimising third-party service provider selection — guiding members to appropriate treatment options while balancing cost, quality, and customer experience:

• Care pathway optimisation: Evaluates service providers based on cost, proximity, urgency, and patient satisfaction
• Transparent recommendations: Presents prioritised service provider options with an explainable rationale
• Human-in-the-loop oversight: Claims handlers remain the final decision-makers, using the AI Agent’s analysis to guide customer interactions

“This deployment shows how enterprise AI agents can capture and scale the nuanced reasoning of experienced claims professionals, enabling consistent, high-quality decision-making in regulated industries,” said Karli Kalpala, Head of Strategy and Agentic AI at Digital Workforce. “Rather than personal assistants or copilots, we focus on enterprise-grade digital colleagues that handle complex work across the enterprise. Real value comes from designing AI as part of the operating model — so it scales reliably, operates under clear governance, and delivers outcomes regulated businesses can trust.”

Production pilot results: factual accuracy, compliance, and user trust
The production pilot, run in late 2025 using real claims data and live operations, delivered strong outcomes. No hallucinations were observed during the pilot, and the AI Agent’s recommendations aligned with established standards, supporting consistent decision quality. The solution was well received by claims professionals as a decision-support tool that improves speed and confidence in customer-facing interactions.

Built for enterprise operations, not consumer-style AI
Unlike traditional consumer AI assistants and chatbots, the AI Agent operates as an enterprise-grade digital colleague:

• Executes multi-step workflows across data sources and systems
• Provides explainable, auditable reasoning behind each recommendation
• Handles real-world variation and incomplete information with resilience
• Integrates into existing claims infrastructure to enhance core processes

The deployment demonstrates how regulated insurers can safely move beyond experimentation and embed AI agents into core decision-making processes at scale.

For more information, please contact
Karli Kalpala, Head of Strategy and Agentic AI Business, Digital Workforce Services Plc,
karli.kalpala@digitalworkforce.com

About Digital Workforce Services Plc
Digital Workforce Services Plc (Nasdaq First North: DWF) is a leader in business automation and technology solutions. With the Digital Workforce Outsmart platform and services—including Enterprise AI agents—organizations transform knowledge work, reduce costs, accelerate digitization, grow revenue, and improve customer experience. More than 200 large customers use our services to drive the transformation of work through automation and Agentic AI. Digital Workforce has particularly strong experience in healthcare, automating care pathways across clinical and administrative workflows to reduce burden, enhance patient safety, and return time to patient care. Following the acquisition of e18 Innovation, the company has further strengthened its position in the UK healthcare pathway automation. We focus on repeatable, outcome-based use cases, and we operate with high integrity and close customer collaboration.Founded in 2015, Digital Workforce employs more than 200 automation professionals in the US, UK, Ireland, and Northern and Central Europe. Our vision: Transforming Work – Beyond Productivity.
https://digitalworkforce.com |https://agent-workforce.com

The post Major Insurer and Digital Workforce Launch AI Agent for Personal Injury Claims, With Zero Hallucinations Observed in Production Pilot appeared first on Digital Workforce.

In the lifecycle of every Small to Medium Enterprise (SME), there is a recurring bottleneck: The Intelligence Gap. As a founder or lead engineer, you need real-time market data, competitor analysis, and technical trend reports to stay competitive. However, hiring a dedicated research team adds significant overhead, while manual research drains high-value engineering hours.

We have officially moved past the era of "Chatting with AI." We are now in the Agentic Era. This tutorial will show you how to build a fully autonomous Multi-Agent Research Squad that lives inside your Google Workspace. By the end of this guide, you will have a system that scans the web, analyzes data against your specific business logic, and drafts executive briefings—all while you sleep, and with zero additional payroll.

1. The Architecture: Multi-Agent Orchestration

Instead of using one general-purpose prompt, we will use Agentic Workflows. We’ll break a complex task into three specialized personas:

1. The Scout (Search Agent): Uses the Gemini API with Google Search grounding to find "fresh" data.
2. The Analyst (Logic Agent): Processes the Scout’s findings, filtering for relevance and "signal" versus "noise."
3. The Editor (Reporting Agent): Synthesizes the analysis into a polished Google Doc or email.

Why this works: For those familiar with robotics and IoT, think of this as a closed-loop control system. The Scout is your sensor (input), the Analyst is your controller (processing), and the Editor is your actuator (output).

2. The Tech Stack

To keep this "Zero-Overhead," we are leveraging tools you likely already use:

• Google Sheets: Our "State Management" database.
• Google Apps Script: Our orchestration engine (The "Glue").
• Gemini 1.5 Pro API: The reasoning engine (Free tier available for developers).
• Google Docs API: Our final delivery medium.

3. Step-by-Step Implementation

Step 1: Setting up the "State Database"

Create a Google Sheet with the following headers:

• Column A: Research Topic (e.g., "Competitor advancements in Solid-State Batteries")
• Column B: Raw Data (Scout’s Output)
• Column C: Analysis (Analyst’s Output)
• Column D: Status (Pending/Complete)

Step 2: Provisioning the Brains

Head to the Google AI Studio and generate an API key for Gemini 1.5 Pro. This model is preferred for its massive context window, allowing it to "read" entire competitor whitepapers if necessary.

Step 3: The Orchestration Script (The Glue)

Open your Google Sheet, go to Extensions > Apps Script, and replace the editor with the logic below. This script acts as the "Manager" that calls the agents in sequence.

JavaScript

const GEMINI_API_KEY = 'YOUR_API_KEY_HERE';

const GEMINI_URL = `https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-pro:generateContent?key=${GEMINI_API_KEY}`;

function runResearchSquad() {

  const sheet = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();

  const data = sheet.getDataRange().getValues();

  // Iterate through rows where status is "Pending"

  for (let i = 1; i < data.length; i++) {

    if (data[i][3] === "Pending") {

      const topic = data[i][0];

      // 1. Call The Scout

      const rawData = callGeminiAgent(topic, "You are a Scout. Find the 5 most recent technical breakthroughs in this field. Return summaries and URLs.");

      sheet.getRange(i + 1, 2).setValue(rawData);

      // 2. Call The Analyst

      const analysis = callGeminiAgent(rawData, "You are a Senior Technical Analyst. Evaluate this data for SME market impact. What are the risks and opportunities?");

      sheet.getRange(i + 1, 3).setValue(analysis);

      // 3. Mark as Complete

      sheet.getRange(i + 1, 4).setValue("Complete");

      // 4. Trigger The Editor (Optional: Send Email)

      sendExecutiveBriefing(topic, analysis);

    }

  }

}

function callGeminiAgent(input, systemInstruction) {

  const payload = {

    "contents": [{

      "parts": [{

        "text": `${systemInstruction}\n\nInput Data: ${input}`

      }]

    }]

  };

  const options = {

    "method": "post",

    "contentType": "application/json",

    "payload": JSON.stringify(payload)

  };

  const response = UrlFetchApp.fetch(GEMINI_URL, options);

  const json = JSON.parse(response.getContentText());

  return json.candidates[0].content.parts[0].text;

}

4. Engineering the Persona Prompts

The secret sauce of a multi-agent system is the System Instruction. To get high-level output, you must define the "Agent's" constraints:

• The Scout's Prompt: "You are a specialized Web Crawler. Your goal is to bypass marketing fluff and find technical specifications, pricing changes, or patent filings. Be concise."
• The Analyst's Prompt: "You are a McKinsey-level strategist. Contrast the Scout's data against a standard SME budget. Identify 'Low-Hanging Fruit' for implementation."

5. Deployment: The "Set and Forget" Strategy

In Apps Script, click the Triggers (Clock Icon) on the left sidebar. Set runResearchSquad to run:

• Time-driven
• Weekly Timer
• Every Monday, 6:00 AM

Now, every Monday morning, your "Squad" will have scanned the market, analyzed the data, and updated your sheet (or emailed your briefing) before you even open your laptop.

The Strategic Edge

For a SME, this isn't just about saving time; it's about asymmetric capability. By automating the intelligence-gathering phase, you allow your human staff to focus entirely on execution. You are effectively running a "Deep Research" department for the cost of a few API calls.

As we continue to explore the intersection of AI, Robotics, and ML, the most successful entities will be those that view AI not as a tool, but as a workforce.

For more deep dives into autonomous AI orchestration and the future of quantum-ready intelligence, stay tuned to AI Quantum Intelligence. We provide the blueprints for the next generation of tech leaders.

Written/published by Kevin Marshall with the help of AI models (AI Quantum Intelligence).

Evaluating OCR systems that convert PDFs or document images into Markdown is far more complex than it appears. Unlike plain text OCR, OCR-to-Markdown requires models to recover content, layout, reading order, and representation choices simultaneously. Today’s benchmarks attempt to score this with a mix of string matching, heuristic alignment, and format-specific rules—but in practice, these approaches routinely misclassify correct outputs as failures.

This post outlines why OCR-to-Markdown evaluation is inherently underspecified, examines common evaluation techniques and their failure modes, highlights concrete issues observed in two widely used benchmarks, and explains why LLM-as-judge is currently the most practical way to evaluate these systems—despite its imperfections .

Why OCR-to-Markdown Is Hard to Evaluate

At its core, OCR-to-Markdown does not have a single correct output.

Multiple outputs can be equally valid:

• Multi-column layouts can be linearized in different reading orders.
• Equations can be represented using LaTeX, Unicode, HTML, or hybrids.
• Headers, footers, watermarks, and marginal text may or may not be considered “content” depending on task intent.
• Spacing, punctuation, and Unicode normalization often differ without affecting meaning.

From a human or downstream-system perspective, these outputs are equivalent. From a benchmark’s perspective, they often are not.

Common Evaluation Techniques and Their Limitations

1. String-Based Metrics (Edit Distance, Exact Match)

Most OCR-to-Markdown benchmarks rely on normalized string comparison or edit distance.

Limitations

• Markdown is treated as a flat character sequence, ignoring structure.
• Minor formatting differences produce large penalties.
• Structurally incorrect outputs can score well if text overlaps.
• Scores correlate poorly with human judgment.

These metrics reward formatting compliance rather than correctness.

2. Order-Sensitive Block Matching

Some benchmarks segment documents into blocks and score ordering and proximity.

Limitations

• Valid alternative reading orders (e.g., multi-column documents) are penalized.
• Small footer or marginal text can break strict ordering constraints.
• Matching heuristics degrade rapidly as layout complexity increases.

Correct content is often marked wrong due to ordering assumptions.

3. Equation Matching via LaTeX Normalization

Math-heavy benchmarks typically expect equations to be rendered as complete LaTeX.

Limitations

• Unicode or partially rendered equations are penalized.
• Equivalent LaTeX expressions using different macros fail to match.
• Mixed LaTeX/Markdown/HTML representations are not handled.
• Rendering-correct equations still fail string-level checks.

This conflates representation choice with mathematical correctness.

4. Format-Specific Assumptions

Benchmarks implicitly encode a preferred output style.

Limitations

• HTML tags (e.g., ) cause matching failures.
• Unicode symbols (e.g., km²) are penalized against LaTeX equivalents.
• Spacing and punctuation inconsistencies in ground truth amplify errors.

Models aligned to benchmark formatting outperform more general OCR systems.

Issues Observed in Existing Benchmarks

Benchmark A: olmOCRBench

Manual inspection reveals that several subsets embed implicit content omission rules:

• Headers, footers, and watermarks that are visibly present in documents are explicitly marked as absent in ground truth.
• Models trained to extract all visible text are penalized for being correct.
• These subsets effectively evaluate selective suppression, not OCR quality.

Additionally:

• Math-heavy subsets fail when equations are not fully normalized LaTeX.
• Correct predictions are penalized due to representation differences.

As a result, scores strongly depend on whether a model’s output philosophy matches the benchmark’s hidden assumptions.

Example 1

For the above image, Nanonets-OCR2 correctly predicts the watermark to the right side of the image, but in the ground truth annotation penalizes the model for predicting it correctly.

{
"pdf": "headers_footers/ef5e1f5960b9f865c8257f9ce4ff152a13a2559c_page_26.pdf", 
"page": 1, 
"id": "ef5e1f5960b9f865c8257f9ce4ff152a13a2559c_page_26.pdf_manual_01", 
"type": "absent", 
"text": "Document t\\u00e9l\\u00e9charg\\u00e9 depuis www.cairn.info - Universit\\u00e9 de Marne-la-Vall\\u00e9e - - 193.50.159.70 - 20/03/2014 09h07. \\u00a9 S.A.C.", "case_sensitive": false, "max_diffs": 3, "checked": "verified", "first_n": null, "last_n": null, "url": ""}

Type absent means that in the prediction data, that text should not be present.

Example 2

The benchmark also does not consider texts that are present in the document footer.

Example in this document, the Alcoholics Anonymous\\u00ae and www.aa.org should not be present in the document according to the ground-truth, which is incorrect

{
	"pdf": "headers_footers/3754542bf828b42b268defe21db8526945928834_page_4.pdf", 
	"page": 1, 
	"id": "3754542bf828b42b268defe21db8526945928834_page_4_header_00", 
	"type": "absent", 
	"max_diffs": 0, 
	"checked": "verified", 
	"url": "", 
	"text": "Alcoholics Anonymous\\u00ae", 
	"case_sensitive": false, "first_n": null, "last_n": null
	}
{
	"pdf": "headers_footers/3754542bf828b42b268defe21db8526945928834_page_4.pdf", 
	"page": 1, 
	"id": "3754542bf828b42b268defe21db8526945928834_page_4_header_01", 
	"type": "absent", 
	"max_diffs": 0, 
	"checked": "verified", 
	"url": "", 
	"text": "www.aa.org", 
	"case_sensitive": false, "first_n": null, "last_n": null}

Benchmark B: OmniDocBench

OmniDocBench exhibits similar issues, but more broadly:

• Equation evaluation relies on strict LaTeX string equivalence.
• Semantically identical equations fail due to macro, spacing, or symbol differences.
• Numerous ground-truth annotation errors were observed (missing tokens, malformed math, incorrect spacing).
• Unicode normalization and spacing differences systematically reduce scores.
• Prediction selection heuristics can fail even when the correct answer is fully present.

In many cases, low scores reflect benchmark artifacts, not model errors.

Example 1

In the example above, the Nanonets-OCR2-3B predicts 5 g silica + 3 g Al$_2$O$_3$ but the ground truth expects as $ 5g \\\\mathrm{\\\\ s i l i c a}+3g \\\\mathrm{\\\\ A l}*{2} \\\\mathrm{O*{3}} $ . This flags the model prediction as incorrect, even when both are correct.

Complete Ground Truth and Prediction, and the test case shared below:

'pred': 'The collected eluant was concentrated by rotary evaporator to 1 ml. The extracts were finally passed through a final column filled with 5 g silica + 3 g Al$_2$O$_3$ to remove any co-extractive compounds that may cause instrumental interferences durin the analysis. The extract was eluted with 120 ml of DCM:n-hexane (1:1), the first 18 ml of eluent was discarded and the rest were collected, which contains the analytes of interest. The extract was exchanged into n-hexane, concentrated to 1 ml to which 1 μg/ml of internal standard was added.'
'gt': 'The collected eluant was concentrated by rotary evaporator to 1 ml .The extracts were finally passed through a final column filled with $ 5g \\\\mathrm{\\\\ s i l i c a}+3g \\\\mathrm{\\\\ A l}*{2} \\\\mathrm{O*{3}} $ to remove any co-extractive compounds that may cause instrumental
interferences during the analysis. The extract was eluted with 120 ml of DCM:n-hexane (1:1), the first 18 ml of eluent was discarded and the rest were collected, which contains the analytes of interest. The extract was exchanged into n - hexane, concentrated to 1 ml to which $ \\\\mu\\\\mathrm{g / ml} $ of internal standard was added.'

Example 2

We found significantly more incorrect annotations with OmniDocBench

In the ground-truth annotation 1 is missing in 1 ml .

'text': 'The collected eluant was concentrated by rotary evaporator to 1 ml .The extracts were finally passed through a final column filled with $ 5g \\\\mathrm{\\\\ s i l i c a}+3g \\\\mathrm{\\\\ A l}*{2} \\\\mathrm{O*{3}} $ to remove any co-extractive compounds that may cause instrumental interferences during the analysis. The extract was eluted with 120 ml of DCM:n-hexane (1:1), the first 18 ml of eluent was discarded and the rest were collected, which contains the analytes of interest. The extract was exchanged into n - hexane, concentrated to 1 ml to which $ \\\\mu\\\\mathrm{g / ml} $ of internal standard was added.'

Enterprises today face a data paradox: while information is abundant, actionable, structured data is scarce. This challenge is a major bottleneck for AI agents and large language models (LLMs). Automated data extraction solves this by acting as the input layer for every AI-driven workflow. It programmatically converts raw data—from documents, APIs, and web pages—into a consistent, machine-readable format, enabling AI to act intelligently.

The reality, however, is that many organizations still depend on manual data wrangling. Analysts retype vendor invoice details into ERP systems, ops staff download and clean CSV exports, and compliance teams copy-paste content from scanned PDFs into spreadsheets. Manual data wrangling creates two serious risks: slow decision-making and costly errors that ripple through downstream automations or cause model hallucinations.

Automation solves these problems by delivering faster, more accurate, and more scalable extraction. Systems can normalize formats, handle diverse inputs, and flag anomalies far more consistently than human teams. Data extraction is no longer an operational afterthought — it’s an enabler of analytics, compliance, and now, intelligent automation.

This guide explores that enabler in depth. From different data sources (structured APIs to messy scanned documents) to extraction techniques (regex, ML models, LLMs), we’ll cover the methods and trade-offs that matter. We’ll also examine agentic workflows powered by extraction and how to design a scalable data ingestion layer for enterprise AI.

What Is Automated Data Extraction?

If data extraction is the first domino in AI automation, then automated data extraction is the mechanism that makes that domino fall consistently, at scale. At its core, it refers to the programmatic capture and conversion of information from any source into structured, machine-usable formats — with minimal human intervention.

Think of extraction as the workhorse behind ingestion pipelines: while ingestion brings data into your systems, extraction is the process that parses, labels, and standardizes raw inputs—from PDFs or APIs—into structured formats ready for downstream use. Without clean outputs from extraction, ingestion becomes a bottleneck and compromises automation reliability.

Unlike manual processes where analysts reformat spreadsheets or copy values from documents, automated extraction systems are designed to ingest data continuously and reliably across multiple formats and systems.

? The Source Spectrum of Data Extraction

Not all data looks the same, and not all extraction methods are equal. In practice, enterprises encounter four broad categories:

• Structured sources — APIs, relational databases, CSVs, SQL-based finance ledgers or CRM contact lists where information already follows a schema. Extraction here often means standardizing or syncing data rather than deciphering it.
• Semi-structured sources — XML or JSON feeds, ERP exports, or spreadsheets with inconsistent headers. These require parsing logic that can adapt as structures evolve.
• Unstructured sources — PDFs, free-text emails, log files, web pages, and even IoT sensor streams. These are the most challenging, often requiring a mix of NLP, pattern recognition, and ML models to make sense of irregular inputs.
• Documents as a special case — These combine layout complexity and unstructured content, requiring specialized methods. Covered in depth later.

? Strategic Goals of Automation

Automated data extraction isn’t just about convenience — it’s about enabling enterprises to operate at the speed and scale demanded by AI-led automation. The goals are clear:

• Scalability — handle millions of records or thousands of files without linear increases in headcount.
• Speed — enable real-time or near-real-time inputs for AI-driven workflows.
• Accuracy — reduce human error and ensure consistency across formats and sources.
• Reduced manual toil — free up analysts, ops, and compliance staff from repetitive, low-value data tasks.

When these goals are achieved, AI agents stop being proof-of-concept demos and start becoming trusted systems of action.

Data Types and Sources — What Are We Extracting From?

Defining automated data extraction is one thing; implementing it across the messy reality of enterprise systems is another. The challenge isn’t just volume — it’s variety.

Data hides in databases, flows through APIs, clogs email inboxes, gets trapped in PDFs, and is emitted in streams from IoT sensors. Each of these sources demands a different approach, which is why successful extraction architectures are modular by design.

?️ Structured Systems

Structured data sources are the most straightforward to extract from because they already follow defined schemas. Relational databases, CRM systems, and APIs fall into this category.

• Relational DBs: A financial services firm might query a Postgres database to extract daily FX trade data. SQL queries and ETL tools can handle this at scale.
• APIs: Payment providers like Stripe or PayPal expose clean JSON payloads for transactions, making extraction almost trivial.
• CSV exports: BI platforms often generate CSV files for reporting; extraction is as simple as ingesting these into a data warehouse.

Here, the extraction challenge isn’t technical parsing but data governance — ensuring schemas are consistent across systems and time.

? Semi-Structured Feeds

Semi-structured sources sit between predictable and chaotic. They carry some organization but lack rigid schemas, making automation brittle if formats change.

• ERP exports: A NetSuite or SAP export might contain vendor payment schedules, but field labels vary by configuration.
• XML/JSON feeds: E-commerce sites send order data in JSON, but new product categories or attributes appear unpredictably.
• Spreadsheets: Sales teams often maintain Excel files where some columns are consistent, but others differ regionally.

Extraction here often relies on parsers (XML/JSON libraries) combined with machine learning for schema drift detection. For example, an ML model might flag that “supplier_id” and “vendor_number” refer to the same field across two ERP instances.

? Unstructured Sources

Unstructured data is the most abundant — and the most difficult to automate.

• Web scraping: Pulling competitor pricing from retail sites requires HTML parsing, handling inconsistent layouts, and bypassing anti-bot systems.
• Logs: Cloud applications generate massive logs in formats like JSON or plaintext, but schemas evolve constantly. Security logs today may include fields that didn’t exist last month, complicating automated parsing.
• Emails and chats: Customer complaints or support tickets rarely follow templates; NLP models are needed to extract intents, entities, and priorities.

The biggest challenge is context extraction. Unlike structured sources, the meaning isn’t obvious, so NLP, classification, and embeddings often supplement traditional parsing.

? Documents as a Specialized Subset

Documents deserve special attention within unstructured sources. Invoices, contracts, delivery notes, and medical forms are common enterprise inputs but combine text, tables, signatures, and checkboxes.

• Invoices: Line items may shift position depending on vendor template.
• Contracts: Key terms like “termination date” or “jurisdiction” hide in free text.
• Insurance forms: Accident claims may include both handwriting and printed checkboxes.

Extraction here typically requires OCR + layout-aware models + business rules validation. Platforms like Nanonets specialize in building these document pipelines because generic NLP or OCR alone often falls short.

? Why Modularity Matters

No single technique can handle all of these sources. Structured APIs might be handled with ETL pipelines, while scanned documents require OCR, and logs demand schema-aware streaming parsers. Enterprises that try to force-fit one approach quickly hit failure points.

Instead, modern architectures deploy modular extractors — each tuned to its source type, but unified through common validation, monitoring, and integration layers. This ensures extraction isn’t just accurate in isolation but also cohesive across the enterprise.

Automated Data Extraction Techniques — From Regex to LLMs

Knowing where data resides is only half the challenge. The next step is understanding how to extract it. Extraction methods have evolved dramatically over the last two decades — from brittle, rule-based scripts to sophisticated AI-driven systems capable of parsing multimodal sources. Today, enterprises often rely on a layered toolkit that combines the best of traditional, machine learning, and LLM-based approaches.

?️ Traditional Methods: Rules, Regex, and SQL

In the early days of enterprise automation, extraction was handled primarily through rule-based parsing.

• Regex (Regular Expressions): A common technique for pulling patterns out of text. For example, extracting email addresses or invoice numbers from a body of text. Regex is precise but brittle — small format changes can break the rules.
• Rule-based parsing: Many ETL (Extract, Transform, Load) systems depend on predefined mappings. For example, a bank might map “Acct_Num” fields in one database to “AccountID” in another.
• SQL queries and ETL frameworks: In structured systems, extraction often looks like running a SQL query to pull records from a database, or using an ETL framework (Informatica, Talend, dbt) to move and transform data at scale.
• Web scraping: For semi-structured HTML, libraries like BeautifulSoup or Scrapy allow enterprises to extract product prices, stock levels, or reviews. But as anti-bot protections advance, scraping becomes fragile and resource-intensive.

These approaches are still relevant where structure is stable — for example, extracting fixed-format financial reports. But they lack flexibility in dynamic, real-world environments.

? ML-Powered Extraction: Learning Patterns Beyond Rules

Machine learning brought a step-change by allowing systems to learn from examples instead of relying solely on brittle rules.

• NLP & NER models: Named Entity Recognition (NER) models can identify entities like names, dates, addresses, or amounts in unstructured text. For instance, parsing resumes to extract candidate skills.
• Structured classification: ML classifiers can label sections of documents (e.g., “invoice header” vs. “line item”). This allows systems to adapt to layout variance.
• Document-specific pipelines: Intelligent Document Processing (IDP) platforms combine OCR + layout analysis + NLP. A typical pipeline:
  • OCR extracts raw text from a scanned invoice.
  • Layout models detect bounding boxes for tables and fields.
  • Business rules or ML models label and validate key-value pairs.

Intelligent Document Processing (IDP) platforms illustrate how this approach combines deterministic rules with ML-driven methods to extract data from highly variable document formats.

The advantage of ML-powered methods is adaptability. Instead of hand-coding patterns, you train models on examples, and they learn to generalize. The trade-off is the need for training data, feedback loops, and monitoring.

? LLM-Enhanced Extraction: Language Models as Orchestrators

With the rise of large language models, a new paradigm has emerged: LLMs as extraction engines.

• Prompt-based extraction: By carefully designing prompts, you can instruct an LLM to read a block of text and return structured JSON (e.g., “Extract all product SKUs and prices from this email”). Tools like LangChain formalize this into workflows.
• Function-calling and tool use: Some LLMs support structured outputs (e.g., OpenAI’s function-calling), where the model fills defined schema slots. This makes the extraction process more predictable.
• Agentic orchestration: Instead of just extracting, LLMs can act as controllers — deciding whether to parse directly, call a specialized parser, or flag low-confidence cases for human review. This blends flexibility with guardrails.

LLMs shine when handling long-context documents, free-text emails, or heterogeneous data sources. But they require careful design to avoid “black-box” unpredictability. Hallucinations remain a risk. Without grounding, LLMs might fabricate values or misinterpret formats. This is especially dangerous in regulated domains like finance or healthcare.

? Hybrid Architectures: Best of Both Worlds

The most effective modern systems today rarely choose one technique. Instead, they adopt hybrid architectures:

• LLMs + deterministic parsing: An LLM routes the input — e.g., detecting whether a file is an invoice, log, or API payload — and then hands off to the appropriate specialized extractor (regex, parser, or IDP).
• Validation loops: Extracted data is validated against business rules (e.g., “Invoice totals must equal line-item sums”, or “e-commerce price fields must fall within historical ranges”).
• Human-in-the-loop: Low-confidence outputs are escalated to human reviewers, and their corrections feed back into model retraining.

This hybrid approach maximizes flexibility without sacrificing reliability. It also ensures that when agents consume extracted data, they’re not relying blindly on a single, failure-prone method.

⚡ Why This Matters for Enterprise AI

For AI agents to act autonomously, their perception layer must be robust.

Regex alone is too rigid, ML alone may struggle with edge cases, and LLMs alone can hallucinate. But together, they form a resilient pipeline that balances precision, adaptability, and scalability.

Among all these sources, documents remain the most error-prone and least predictable — demanding their own extraction playbook.

Deep Dive — Document Data Extraction

Of all the data sources enterprises face, documents are consistently the hardest to automate. Unlike APIs or databases with predictable schemas, documents arrive in thousands of formats, riddled with visual noise, layout quirks, and inconsistent quality. A scanned invoice may look different from one vendor to another, contracts may hide critical clauses in dense paragraphs, and handwritten notes can throw off even the most advanced OCR systems.

⚠️ Why Documents Are So Hard to Extract From

1. Layout variability: No two invoices, contracts, or forms look the same. Fields shift position, labels change wording, and new templates appear constantly.
2. Visual noise: Logos, watermarks, stamps, or handwritten notes complicate recognition.
3. Scanning quality: Blurry, rotated, or skewed scans can degrade OCR accuracy.
4. Multimodal content: Documents often combine tables, paragraphs, signatures, checkboxes, and images in the same file.

These factors make documents a worst-case scenario for rule-based or template-based approaches, demanding more adaptive pipelines.

? The Typical Document Extraction Pipeline

Modern document data extraction follows a structured pipeline:

1. OCR (Optical Character Recognition): Converts scanned images into machine-readable text.
2. Layout analysis: Detects visual structures like tables, columns, or bounding boxes.
3. Key-value detection: Identifies semantic pairs such as “Invoice Number → 12345” or “Due Date → 30 Sept 2025.”
4. Validation & human review: Extracted values are checked against business rules (e.g., totals must match line items) and low-confidence cases are routed to humans for verification.

This pipeline is robust, but it still requires ongoing monitoring to keep pace with new document templates and edge cases.

? Advanced Models for Context-Aware Extraction

To move beyond brittle rules, researchers have developed vision-language models that combine text and layout understanding.

• LayoutLM, DocLLM, and related models treat a document as both text and image, capturing positional context. This allows them to understand that a number inside a table labeled “Quantity” means something different than the same number in a “Total” row.
• Vision-language transformers can align visual features (shapes, boxes, logos) with semantic meaning, improving extraction accuracy in noisy scans.

These models don’t just “read” documents — they interpret them in context, a major leap forward for enterprise automation.

? Self-Improving Agents for Document Workflows

The frontier in document data extraction is self-improving agentic systems. Recent research explores combining LLMs + reinforcement learning (RL) to create agents that:

• Attempt extraction.
• Evaluate confidence and errors.
• Learn from corrections over time.

In practice, this means every extraction error becomes training data. Over weeks or months, the system improves automatically, reducing manual oversight.

This shift is critical for industries with high document variability — insurance claims, healthcare, and global logistics — where no static model can capture every possible format.

? Nanonets in Action: Multi-Document Claims Workflows

Document-heavy industries like insurance highlight why specialized extraction is mission-critical. A claims workflow may include:

• Accident report forms (scanned and handwritten).
• Vehicle inspection photos embedded in PDFs.
• Repair shop invoices with line-item variability.
• Policy documents in mixed digital formats.

Nanonets builds pipelines that combine OCR, ML-based layout analysis, and human-in-the-loop validation to handle this complexity. Low-confidence extractions are flagged for review, and human corrections flow back into the training loop. Over time, accuracy improves without requiring rule rewrites for every new template.

This approach enables insurers to process claims faster, with fewer errors, and at lower cost — all while maintaining compliance.

⚡ Why Documents Deserve Their Own Playbook

Unlike structured or even semi-structured data, documents resist one-size-fits-all methods. They require dedicated pipelines, advanced models, and continuous feedback loops. Enterprises that treat documents as “just another source” often see projects stall; those that invest in document-specific extraction strategies unlock speed, accuracy, and downstream AI value.

Real-World AI Workflows That Depend on Automated Extraction

Below are real-world enterprise workflows where AI agents depend on a reliable, structured data extraction layer:

Across these industries, a clear workflow pattern emerges: Extraction → Validation → Agentic Action. The quality of this flow is critical. High-confidence, structured data empowers agents to act autonomously. When confidence is low, the system defers—pausing, escalating, or requesting clarification—ensuring human oversight only where it's truly needed.

This modular approach ensures that agents don’t just consume data, but trustworthy data — enabling speed, accuracy, and scale.

Building a Scalable Automated Data Extraction Layer

All the workflows described above depend on one foundation: a scalable data extraction layer. Without it, enterprises are stuck in pilot purgatory, where automation works for one narrow use case but collapses as soon as new formats or higher volumes are introduced.

To avoid that trap, enterprises must treat automated data extraction as infrastructure: modular, observable, and designed for continuous evolution.

? Build vs Buy: Picking Your Battles

Not every extraction problem needs to be solved in-house. The key is distinguishing between core extraction — capabilities unique to your domain — and contextual extraction, where existing solutions can be leveraged.

• Core examples: A bank developing extraction for regulatory filings, which require domain-specific expertise and compliance controls.
• Contextual examples: Parsing invoices, purchase orders, or IDs — problems solved repeatedly across industries where platforms like Nanonets provide pre-trained pipelines.

A practical strategy is to buy for breadth, build for depth. Use off-the-shelf solutions for commoditized sources, and invest engineering time where extraction quality differentiates your business.

⚙️ Platform Design Principles

A scalable extraction layer is not just a collection of scripts — it’s a platform. Key design elements include:

• API-first architecture: Every extractor (for documents, APIs, logs, web) should expose standardized APIs so downstream systems can consume outputs consistently.
• Modular extractors: Instead of one monolithic parser, build independent modules for documents, web scraping, logs, etc., orchestrated by a central routing engine.
• Schema versioning: Data formats evolve. By versioning output schemas, you ensure downstream consumers don’t break when new fields are added.
• Metadata tagging: Every extracted record should carry metadata (source, timestamp, extractor version, confidence score) to enable traceability and debugging.

? Resilience: Adapting to Change

Your extraction layer's greatest enemy is schema drift—when formats evolve subtly over time.

• A vendor changes invoice templates.
• A SaaS provider updates API payloads.
• A web page shifts its HTML structure.

Without resilience, these small shifts cascade into broken pipelines. Resilient architectures include:

• Adaptive parsers that can handle minor format changes.
• Fallback logic that escalates unexpected inputs to humans.
• Feedback loops where human corrections are fed back into training datasets for continuous improvement.

This ensures the system doesn’t just work today — it gets smarter tomorrow.

? Observability: See What Your Extraction Layer Sees

Extraction is not a black box. Treating it as such—with data going in and out with no visibility—is a dangerous oversight.

Observability should extend to per-field metrics — confidence scores, failure rates, correction frequency, and schema drift incidents. These granular insights drive decisions around retraining, improve alerting, and help trace issues when automation breaks. Dashboards visualizing this telemetry empower teams to continuously tune and prove the reliability of their extraction layer.

• Confidence scores: Every extracted field should include a confidence estimate (e.g., 95% certain this is the invoice date).
• Error logs: Mis-parsed or failed extractions must be tracked and categorized.
• Human corrections: When reviewers fix errors, those corrections should flow back into monitoring dashboards and retraining sets.

With observability, teams can prioritize where to improve and prove compliance — a necessity in regulated industries.

⚡ Why This Matters

Enterprises can’t scale AI by stitching together brittle scripts or ad hoc parsers. They need an extraction layer that is architected like infrastructure: modular, observable, and continuously improving.

Conclusion

AI agents, LLM copilots, and autonomous workflows might feel like the future — but none of them work without one critical layer: reliable, structured data.

This guide has explored the many sources enterprises extract data from — APIs, logs, documents, spreadsheets, and sensor streams — and the variety of techniques used to extract, validate, and act on that data. From claims to contracts, every AI-driven workflow starts with one capability: reliable, scalable data extraction.

Too often, organizations invest heavily in orchestration and modeling — only to find their AI initiatives fail due to unstructured, incomplete, or poorly extracted inputs. The message is clear: your automation stack is only as strong as your automated data extraction layer.

That’s why extraction should be treated as strategic infrastructure — observable, adaptable, and built to evolve. It’s not a temporary preprocessing step. It’s a long-term enabler of AI success.

Start by auditing where your most critical data lives and where human wrangling is still the norm. Then, invest in a scalable, adaptable extraction layer. Because in the world of AI, automation doesn't start with action—it starts with access.

FAQs

What’s the difference between data ingestion and data extraction in enterprise AI pipelines?

Data ingestion is the process of collecting and importing data from various sources into your systems — whether APIs, databases, files, or streams. Extraction, on the other hand, is what makes that ingested data usable. It involves parsing, labeling, and structuring raw inputs (like PDFs or logs) into machine-readable formats that downstream systems or AI agents can work with. Without clean extraction, ingestion becomes a bottleneck, introducing noise and unreliability into the automation pipeline.

What are best practices for validating extracted data in agent-driven workflows?

Validation should be tightly coupled with extraction — not treated as a separate post-processing step. Common practices include applying business rules (e.g., "invoice totals must match line-item sums"), schema checks (e.g., expected fields or clause presence), and anomaly detection (e.g., flagging values that deviate from norms). Outputs with confidence scores below a threshold should be routed to human reviewers. These corrections then feed into training loops to improve extraction accuracy over time.

How does the extraction layer influence agentic decision-making in production?

The extraction layer acts as the perception system for AI agents. When it provides high-confidence, structured data, agents can make autonomous decisions — such as approving payments or routing claims. But if confidence is low or inconsistencies arise, agents must escalate, defer, or request clarification. In this way, the quality of the extraction layer directly determines whether an AI agent can act independently or must seek human input.

What observability metrics should we track in an enterprise-grade data extraction platform?

Key observability metrics include:

• Confidence scores per extracted field.
• Success and failure rates across extraction runs.
• Schema drift frequency (how often formats change).
• Correction rates (how often humans override automated outputs).These metrics help trace errors, guide retraining, identify brittle integrations, and maintain compliance — especially in regulated domains.

Global Business Services (GBS) has evolved from back-office support to a strategic growth engine. With the shared services market projected at $111.3B by 2025 and global digital transformation spend surpassing $2.5T, GBS is now firmly established as a business-critical enabler.

In an era of economic volatility, rapid tech change, and rising expectations, leaders are prioritizing efficiency, agility, and enterprise value. Here are the top priorities shaping 2026, backed by market data, executive insights, and real-world examples.

1. Elevating GBS to a Strategic Business Partner

76% of GBS units now report to the C-suite, underscoring their role in driving growth and working capital improvements. As Deloitte notes, cost reduction alone is a “deteriorating value proposition.”

Still, only 41% of companies believe their shared services deliver tangible value (BCG). To close the gap, leaders are shifting from transactional KPIs to value-based measures like revenue enablement and decision support. Many are embedding Centers of Excellence in analytics and process improvement directly into business lines, positioning GBS as an indispensable internal consultant.

2. Customer-Centric Service Excellence

73% of GBS organizations rank service quality among their top three priorities (SSON Analytics), second only to cost savings. More than half explicitly identify customer experience as critical.

To deliver, GBS units are rolling out clear SLAs, real-time feedback loops, and even “customer success” roles, borrowing from external service models. Enterprises now expect GBS to function as a true business partner, delivering agility, automation, and actionable insights.

3. Operational Efficiency and Cost Optimization

Efficiency remains the foundation: 90% of organizations cite it as their top driver (Deloitte), with nearly half reporting 20%+ savings from their models. For large enterprises, this translates into tens or hundreds of millions annually.

The focus has shifted to intelligent cost optimization using automation, self-service, and process redesign to reduce waste while improving quality. Leading GBS groups reinvest savings into analytics, insights, and service improvements, creating a cycle of continuous value.

4. Digital Transformation and Intelligent Automation

90% of GBS organizations now play a role in their company’s digital agenda, yet only 20% rate themselves as advanced (SSON).

The shift is toward end-to-end automation: AI-assisted workflows, document processing, and integrated platforms. Starbucks’ Digital Process Automation COE demonstrates the impact—35+ RPA use cases and 15 workflow apps have helped double annual savings growth. For 2026, the priority is scaling enterprise-wide automation and embedding digital into every workflow.

5. Generative AI and Next-Gen Technologies

GenAI adoption has skyrocketed: from 10% in 2023 to 80% experimenting by late 2024, with more than half piloting (SSON).

Early adopters report 54% faster service delivery and 51% higher output quality. Use cases include chatbots, document processing, automated reporting, and predictive analytics. RPA remains the #2 investment area, showing leaders are combining traditional automation with GenAI for a complete toolkit. Nearly 60% of GBS groups are using external partners to accelerate adoption.

6. Expanding Scope and Moving Up the Value Chain

GBS is extending beyond transactional work. Nearly 50% of leaders plan to expand into decision support, research, or end-customer services, with another 35% considering it (SSON).

Portfolios now include data analytics (43%), master data management (50%), tax (43%), statutory reporting (41%), and call center support (33%). Unilever and Procter & Gamble already use GBS for analytics-driven insights, supply chain support, and innovation. For 2026, 77% of GBS units plan scope expansion, with many extending into new geographies and business lines.

7. Global Delivery Model and Location Strategy

85% of enterprises now run on the GBS model (Deloitte), supported by multi-location networks for resilience, cost, and talent.

Nearshoring is accelerating: by 2026, 50% of firms will add hubs in Latin America or Europe, with Mexico, Portugal, and Poland joining India and the Philippines as key centers (SSON). Hybrid models, balancing captive and outsourced delivery are becoming the norm, enabling 24/7 operations, risk mitigation, and global talent access.

Conclusion

As GBS leaders step into 2026, the mandate is clear: deliver efficiency and enterprise value in equal measure. Cost optimization remains vital, but the future belongs to organizations that pair operational excellence with strategic impact, driving growth, agility, and sustainability.

No longer back-office utilities, today’s GBS units serve as digital nerve centers: scaling AI and automation, embedding analytics, expanding scope, and optimizing global delivery. They are customer-centric partners, not just cost managers, ensuring, as Auxis puts it, that “customers come for the price but stay for the value.”

By doubling down on these top 10 priorities—from strategic partnering and service excellence to next-gen tech, talent, and ESG—Shared Services organizations are positioning themselves at the forefront of business evolution. The next chapter promises both disruption and opportunity, and GBS will be pivotal in powering intelligent, agile, and sustainable enterprises worldwide.

Sources:

1. SSON Analytics: State of the Shared Services & Outsourcing Industry Report 2025[2][51]
2. Auxis (via SSON Research): “10 Shared Services Trends Shaping the GBS Industry in 2025.” [8][19]
3. Deloitte: 2025 Global Business Services Survey Findings[52][4]
4. The Hackett Group: Key Issues Study 2024: GBS Priorities[3]
5. EY: “How GBS is driving sustainable business transformation”[48][47]
6. SSON Research: GBS Executive Insights and Quotes[53][27]
7. Starbucks GBS Case: PegaWorld 2024 Presentation[17][18]
8. Deloitte Press Release: GBS Model Impacts (Aug 29, 2025)[54][55]
9. SSON Analytics: Shared Services Market and GBS Targets[1][56]
10. Auxis/SSON: Shared Services Trends on Talent & Locations[29][41]

Picture a mid-sized company handling 1,000–2,000 invoices every month—roughly 250–500 invoices per week. On the surface, this doesn’t sound unmanageable. But at an average of 15–16 minutes per invoice, that volume quickly snowballs into 200–400 staff hours every month spent on repetitive tasks like data entry, coding, and chasing approvals. In practical terms, that’s the equivalent of one to two full-time employees dedicated solely to pushing paper instead of adding strategic value.

Beyond the labor drain, the financial impact is staggering. Studies show that manual invoice processing costs between $15 and $20 per invoice, depending on complexity and error rates. For a business processing 1,500 invoices per month—about 18,000 annually—that translates to $270,000–$360,000 per year spent on AP processing alone. Automation can reduce this cost to as little as $3 per invoice, unlocking $180,000–$300,000 in annual savings.

Time-to-payment is equally concerning. Manual workflows stretch invoice cycle times to 10.9–17.4 days on average, while best-in-class automated processes can shrink that to just 2.8–4 days. The result? Stronger vendor relationships, fewer late-payment penalties, and the ability to capture early-payment discounts.

Then there’s accuracy. Manual systems see error rates of ~1.6% per invoice, with mistakes like duplicate payments compounding over time. Intelligent automation reduces errors by up to 80%, dramatically lowering the cost of rework and compliance risk.

For finance leaders, these numbers highlight a hard truth: manual invoice management is not just inefficient—it’s a silent tax on growth.

This is where invoice automation software enters the picture—transforming invoice management from a slow, manual burden into a streamlined, intelligent process. An automated invoice processing system turns this chaos into clarity.

What is Invoice Automation Software?

At its core, invoice processing automation software is designed to streamline the entire invoice-to-pay workflow. Instead of accounts payable (AP) teams manually entering line items, verifying purchase orders, routing documents for approval, and scheduling payments, automation software digitizes each step—from invoice capture to validation, approval routing, and payment execution.

The foundation of invoice automation is data capture — done in seconds, not minutes —extracting key information such as vendor name, invoice number, line items, tax details, and payment terms from documents. Early systems relied heavily on optical character recognition (OCR), which converts scanned text into machine-readable formats.

But traditional OCR tools are rigid: they require pre-built templates for each invoice format, and even minor changes (like a vendor updating their layout) can break extraction accuracy.

This is where AI-first approaches—often called Intelligent Document Processing (IDP)—fundamentally change the game. Unlike template-based OCR, AI-driven systems learn patterns across invoices, adapt to new formats dynamically, and continuously improve with usage. This allows them to handle invoices from thousands of vendors without requiring constant template maintenance.

Why does this distinction matter? Because at scale, template fragility becomes a bottleneck. A mid-sized company might process invoices from hundreds of suppliers, while enterprises manage tens of thousands. Each vendor may have multiple formats, currencies, or tax codes. In template-based OCR systems, every variation needs manual configuration. With AI-first platforms, invoices are captured accurately regardless of format, enabling AP teams to spend time on exceptions and approvals instead of fixing broken templates. Unlike outdated template-based OCR, these invoice automation solutions ensure accuracy at scale.

Simply put, invoice automation software—especially when powered by AI-first capture—turns a fragmented, error-prone process into a seamless, touchless workflow, allowing businesses to reduce costs, improve accuracy, and scale operations without scaling headcount.

But beyond efficiency, why does this matter so much for businesses today? The answer lies in the very real savings and competitive advantages automation delivers.

Why Businesses Need Invoice Automation

Even in organizations that have digitized other finance functions, AP often remains stubbornly manual—without an automated invoice processing system to streamline workflows. As we saw earlier, processing invoices manually consumes hundreds of staff hours, costs upwards of $15 per invoice, and introduces error risks that undermine accuracy and compliance. Add to that scattered invoices across inboxes and filing cabinets, and the result is poor cash flow visibility and lack of real-time control.

The ripple effects are significant. Companies miss out on early-payment discounts, absorb late fees, struggle with compliance, and strain relationships with vendors. What should be a straightforward operational process becomes a bottleneck that drains working capital and productivity.

Invoice automation flips this equation. By digitizing capture, validation, and approval workflows, organizations dramatically reduce cycle times, cut costs, and improve accuracy. More importantly, automation frees finance teams from repetitive data entry, allowing them to focus on analysis, planning, and supplier strategy.

The benefits are clear:

• Cost savings: Automation reduces invoice costs by more than 80%, unlocking six-figure savings annually for mid-sized firms.
• Speed: Cycle times fall from weeks to just a few days, helping companies avoid late fees and capture early-payment discounts.
• Accuracy: Error rates drop dramatically, cutting duplicate payments and manual rework.
• Capacity: Finance teams free up the equivalent of 1–2 FTEs annually to focus on higher-value tasks.

? Case Study: Asian Paints + Nanonets

One of Asia’s largest paint manufacturers adopted an automatic invoice processing solution to tackle this burden. With Nanonets, they cut invoice processing time from five minutes to ~30 seconds per document—a 90% reduction. By automating extraction and routing into SAP, the company saved 192 hours per month (~10 FTE days) and positioned itself to manage 22,000+ vendors with minimal manual intervention.

? Read the full case study

? Case Study: SaltPay + Nanonets

SaltPay, a fast-growing payments provider, manages over 100,000 vendors. Manual processing was slowing down growth. By integrating Nanonets with SAP, SaltPay achieved near-100% accuracy in data capture and realized 99% time savings compared to manual workflows. Finance teams shifted from invoice coding to supplier management and strategic finance projects, strengthening both throughput and vendor relationships.

? Read the full case study

In short: automation transforms AP from a costly liability into a strategic enabler of cash flow visibility, compliance, and supplier trust.

Must-Have Features of the Best Invoice Automation Software

Once you understand why invoice automation is critical, the next question is obvious: what features separate the best platforms from the rest?

Not all solutions deliver true automation; some still rely heavily on templates, manual intervention, or clunky integrations. The right software should combine intelligence, flexibility, and scalability to fit your business today—and grow with you tomorrow.

These are the non-negotiable features every invoice automation solution should provide:

1. AI-First Data Capture

At the heart of invoice automation lies accurate data extraction. Legacy OCR systems require templates for each invoice layout, making them fragile and maintenance-heavy. A small change in a vendor’s format can break extraction and flood AP teams with exceptions. By contrast, AI-first systems learn invoice layouts without templates. They adapt to new formats dynamically, ensuring high accuracy across thousands of vendors and document types. This is critical for scaling without creating new back-office burdens.

2. Business Rule Validations

Capturing data is only the first step. Best-in-class systems apply business rule validations automatically, ensuring invoices comply with organizational and regulatory requirements before they ever hit approval queues. Examples include:

• 3-way matching (invoice vs. purchase order vs. goods receipt).
• Vendor compliance checks, such as validating supplier bank details against master records.
• Duplicate detection, flagging invoices with the same number or amount already processed.
• Tax and VAT compliance, automatically verifying rates and jurisdiction-specific rules.
• Threshold alerts, flagging invoices above a set amount for additional approval.These rules not only reduce exceptions but also safeguard against fraud and compliance risks.

3. Flexible Approval Workflows

AP processes are rarely linear. Invoices may need multiple reviewers across departments, special handling based on value, or emergency escalation when deadlines loom. Look for platforms with configurable approval workflows that can:

• Route invoices automatically by vendor, department, or spend category.
• Apply role-based and conditional approvals (e.g., invoices >$10K routed to the CFO).
• Escalate overdue approvals to backup reviewers.
• Allow mobile approvals, enabling busy executives to approve on the go.
• Support delegation when an approver is out of office.By automating these workflows, companies eliminate bottlenecks, reduce back-and-forth emails, and keep payment cycles on track.

4. ERP & Accounting Integrations in Invoice Processing Automation Software

Automation only delivers full value if it connects seamlessly to your finance stack. Leading platforms offer native integrations with ERP and accounting systems such as QuickBooks, NetSuite, SAP, and Oracle. This ensures that invoice data, approvals, and payment status flow automatically into your system of record—removing duplicate entry and reducing reconciliation headaches.

5. Analytics & Reporting

Top-tier platforms go beyond processing to deliver visibility and control. Dashboards should track KPIs such as:

• Average cycle time per invoice.
• Exception rates and bottlenecks.
• Spend by vendor or category.
• Percentage of invoices captured and approved touchlessly.

These insights help CFOs and controllers optimize working capital, identify process inefficiencies, and negotiate better vendor terms.

6.Scalability & User Experience

Finally, the platform should grow with your business. That means handling volume spikes gracefully (think quarter-end invoice surges), supporting multi-entity or global structures, and maintaining high accuracy even as complexity increases. Just as important: a clean, intuitive interface. If AP staff find the system clunky, adoption will lag and the value of automation will erode. A strong user experience ensures teams embrace the tool instead of working around it.

Best Invoice Automation Software in 2025

Understanding the must-have features is one thing; finding the right solution is another. The market for invoice automation has exploded, with dozens of vendors promising speed, accuracy, and integration. But not every platform delivers the same value. Some excel at end-to-end AP automation, while others focus on niche strengths like AI-first capture or small business simplicity.

To help you navigate the options, we’ve grouped the leading invoice processing automation software into four categories—each suited to a different business profile:

• End-to-End AP Automation for companies seeking comprehensive control from invoice to payment.
• Small Business Tools for firms that want affordability and ease of use.
• Enterprise ERP Solutions for large organizations needing deep system integration.
• AI-First Extraction Engines for businesses looking to modernize capture without overhauling their ERP stack.

In the sections that follow, we’ll break down each vendor by target use case, key features, pricing, pros and cons, integrations, and ideal customer profile.

? Automated Invoice Processing Software Landscape at a Glance

Now let’s take a closer look at each of these solutions to see how they compare in practice.

a. Best for End-to-End AP Automation (Tipalti & Stampli)

Tipalti

• Target use case: Businesses needing full-spectrum AP—from invoice capture to global payouts—especially where compliance and scalability matter.
• Key features: AI-driven invoice capture; 2-/3-way matching; supplier self-onboarding and built-in tax compliance; global mass payments; real-time reconciliation; spend visibility tools.
• Pricing: SaaS plans starting at $99/month; enterprise pricing on request.
• Pros: Automates global payables; integrates broadly; strong controls.
• Cons: May be overkill for small teams; complexity can be a barrier.
• Integrations: NetSuite; QuickBooks; Acumatica; Dynamics; Sage; SAP Business One; Xero; SAP S/4HANA; Workday; Infor; and popular performance marketing platforms.
• Ideal customer: Mid-market to enterprise firms managing high-volume, cross-border payables.

Stampli

• Target use case: Teams needing quick AP workflow upgrades that don’t disrupt existing ERPs, with heavy emphasis on collaboration and AI assistance.
• Key features: AI assistant (“Billy the Bot”); seamless QuickBooks integration; 2-/3-way PO matching; vendor portal; unified communication; integrated payments including domestic and international options.
• Pricing: Bundled licensing tied to invoice volume and user roles; connector fees may apply.
• Pros: Deploys fast; an "AP-first" solution that integrates with, rather than replaces, a company's existing ERP - reducing friction in change management.
• Cons: Connector fees and bundled pricing may be opaque for small teams.
• Integrations: QuickBooks; NetSuite; Xero; Sage Intacct; Microsoft Dynamics; SAP; Oracle; workflow tools (Slack, Teams); and over 70 other systems.
• Ideal customer: Mid-market finance teams wanting AP automation without ERP rip-and-replace.

b. Best for Small Businesses (QuickBooks Bill Pay & Melio)

QuickBooks Bill Pay

• Target use case: SMBs embedded within the QuickBooks ecosystem (QuickBooks Online or QuickBooks Desktop) seeking basic yet reliable bill payment automation.
• Key features: Invoice capture via upload or email using OCR; batch payments; automated purchase order matching; basic approval workflows; supplier self-service portals; supports ACH/credit/check options (international payments are limited); provides tools for 1099 compliance for US vendors.
• Pricing: Native to QuickBooks subscriptions; available as an add-on.
• Pros: Low friction; aligned with bookkeeping workflows.
• Cons: Limited advanced workflow or AP analytics beyond Small Business needs; lacks the robust, customizable 3-way matching that is standard in more advanced AP automation platforms; approval workflows are less flexible than those offered by dedicated solutions.
• Integrations: Built-in with QuickBooks Online/Advanced.
• Ideal customer: Small businesses using QuickBooks with light-to-moderate AP volume.

Melio

• Target use case: Very small businesses needing intuitive payables and receivables in one, budgeting simplicity with flexibility on fees.
• Key features: Seamless QuickBooks Online sync; free for standard ACH transactions, with monthly fees for premium plans; extended pay terms; simple vendor onboarding; encrypted data and compliance.
• Pricing: Free for standard use; fees apply for expedited or credit-based payments.
• Pros: Friendly UX; affordable; extended liquidity options.
• Cons: Limited P2P or procurement features.
• Integrations: QuickBooks Online; QuickBooks Desktop; Xero; and FreshBooks, with an open API for custom integrations.
• Ideal customer: Micro-businesses or solo operators seeking pay-on-demand flexibility.

c. Best for Enterprise ERP Workflows (SAP Concur & Coupa)

SAP Concur

• Target use case: Large and global enterprises combining travel, expense, and invoice management under one compliant ecosystem.
• Key features: Automated invoice capture (paper, email, fax) with ML/OCR; mobile expense/receipt matching; real-time spend visibility; AI fraud detection and policy enforcement (Joule AI Copilot); comprehensive analytics.
• Pricing: Custom pricing (~$9/user/month baseline, with quotes scaling up); large footprints likely in five-figure SaaS budgets.
• Pros: Deep coverage across T&E, invoicing, compliance; powerful analytics; ability to enforce policies and provide a single source of truth for all employee-initiated spend.
• Cons: Steeper learning curve; clunky UX; expensive setup and scaling.
• Integrations: NetSuite; SAP ERP (S/4HANA, ECC); Oracle; Microsoft; QuickBooks; HR systems; reporting tools; and a wide ecosystem of hundreds of third-party apps.
• Ideal customer: Global enterprises needing end-to-end spend visibility and governance.

Coupa

• Target use case: Enterprises looking for advanced invoice/PO capabilities, AI validation, vendor collaboration, and rich business spend management (procurement, invoicing, payments, and supply chain management).
• Key features: AI-powered invoice validation; 2- and 3-way matching; e-invoicing; supplier self-service; multi-currency/multi-country handling; optimized payment scheduling; mobile access; dashboards.
• Pricing: Quote-based, often in ~$90K/year mid-tier range.
• Pros: Strong AI and fraud tools; unified view of all spend, powered by AI to automate tasks, improve compliance, and drive savings; scalable.
• Cons: High cost; supplier adoption may require extra change management.
• Integrations: Deep ERP connectors with SAP; Oracle; plus APIs for custom use.
• Ideal customer: Large, often global, enterprise matrixed organizations needing full-suite spend intelligence.

d. Best for AI-First Invoice Extraction (Nanonets & Rossum)

Nanonets

• Target use case: Businesses seeking a nimble, AI-native (Intelligent Document Processing) capture layer that can inject automation into existing systems.
• Key features: Template-free AI OCR customization; integrations with QuickBooks, Xero, and other accounting and ERP systems; highly accurate field extraction; cost-effective for high volumes of invoices; automates 2- and 3-way matching and flags anomalies or duplicate invoices; offers features that support compliance and audit readiness.
• Pricing: Flexible, usage-based pricing with transparent costs.
• Pros: Fast ROI; flexible deployment; accuracy gains.
• Cons: Requires pairing with workflows or ERP to complete automation; not a full-suite AP automation or ERP system with native payment and reconciliation capabilities.
• Integrations: Native integrations with popular accounting software (QuickBooks, Xero, FreshBooks) and robust API connectors for deeper ERP integration (NetSuite, SAP, etc.).
• Ideal customer: Mid-sized firms and enterprises needing smarter capture without full suite commitment.

Rossum

• Target use case: Organizations that already have AP workflows but need more resilient, AI-based invoice data capture capabilities.
• Key features: AI-driven document understanding; customizable templates; validation rules; cloud extraction; real-time dashboards.
• Pricing: Quote-based, with tiered plans starting at a high price point ($18,000 per year).
• Pros: Best-in-class capture; easy integration with existing DMS/ERP.
• Cons: Limited end-to-end AP capabilities; must be layered into existing stack.
• Integrations: API-friendly with native integrations for major ERPs (SAP, Oracle, Coupa) and a wide range of accounting and automation tools.
• Ideal customer: Teams wanting best-in-class capture in place of brittle OCR systems.

How to Choose the Right Invoice Automation Software

With so many options on the market, the question isn’t whether to automate invoices—it’s which platform best fits your business needs. Choosing the right solution requires balancing scale, complexity, and organizational priorities.

Here’s a step-by-step framework to guide evaluation:

Step 1: Assess Invoice Volume and Workflow Complexity

The size of your AP workload is the single most important determinant. A company processing 200 invoices per month has very different needs than one handling 20,000+ invoices globally. Consider not just volume, but also workflow complexity: multi-entity structures, global vendors, tax/VAT rules, or multi-level approval chains.

Step 2: Map to Vendor Categories

Map your workload to the right invoice automation solution (as summarized in the previous section):

• Small Business Tools → Ideal if you process fewer than 500 invoices/month and want low-cost simplicity.
• AI-First or Mid-Market Suites → Best fit for firms handling 1,000–2,000 invoices/month and needing workflow automation with ERP integration.
• Enterprise ERP/Global Suites → Necessary for organizations processing 10,000+ invoices/month, with complex compliance and multi-entity requirements.

Step 3: Consider Persona-Based Priorities

Different stakeholders weigh different factors:

• CFO → Cash visibility, compliance, auditability, ROI.
• Head of Operations → Efficiency, scalability, process resilience.
• AP Manager → Usability, accuracy, ease of onboarding staff.

A successful choice satisfies all three lenses, not just one.

Step 4: Apply a Quick Evaluation Checklist

Before issuing RFPs or scheduling demos, use this five-point filter:

1. Volume fit: Can it handle your current and future invoice load?
2. Integrations: Does it natively connect to your ERP/accounting system?
3. Approval workflows: Are they configurable to your structure?
4. Compliance & security: Does it meet SOC 2, GDPR, SOX, and audit requirements?
5. Budget alignment: Is pricing transparent, and does ROI justify the spend?

In short: choosing invoice automation software is about fit, not flash. By mapping your invoice volume, aligning with vendor categories, considering persona-driven needs, and applying a structured checklist, you can confidently narrow the field to a shortlist that will deliver impact today and scale tomorrow.

Conclusion: Automating Today, Future-Proofing Finance

Invoice automation is no longer just about reducing data entry. The technology is evolving rapidly, and the next wave of innovation is set to redefine how accounts payable functions within modern finance organizations.

Emerging Trends to Watch

• Touchless AP → The holy grail is a fully automated, “straight-through” process where invoices move from capture to validation, approval, and payment with zero human intervention. Early adopters already report significant cycle time reductions, and the expectation is that touchless AP will become the standard rather than the exception.
• Predictive Analytics → With historical invoice data feeding into AI models, businesses will gain the ability to forecast spend, anticipate cash flow requirements, and identify anomalies before they become problems. This shifts AP from a reactive function to a forward-looking partner in financial strategy.
• AI-Led Fraud Detection → Fraudulent invoices, duplicate submissions, and suspicious vendor activity remain a persistent risk. Emerging platforms are embedding machine learning to flag these anomalies in real time, reducing financial leakage and strengthening compliance.
• AI Agents in Finance → Traditional automation tools like RPA were built for repetitive, rules-based tasks, but they break down when workflows involve exceptions or context. The next leap is AI agents—autonomous, goal-driven systems that can reason, adapt, and collaborate with humans. In AP, these agents can negotiate exceptions with suppliers, learn new vendor rules dynamically, route invoices intelligently, and trigger downstream ERP actions without explicit prompts. Early adopters report 65–75% reductions in manual intervention, with agents taking over approvals, compliance checks, and anomaly detection—making AP not just faster, but smarter and more resilient.

Strategic Impact on Finance

As automation matures, accounts payable will no longer be seen as a cost center. Instead, it will become a finance intelligence hub—a source of real-time insights into cash flow, vendor risk, and working capital trends. The biggest shift is cultural: AP teams move from chasing invoices to influencing strategic finance decisions, from liquidity planning to supplier negotiations.

AI agents will accelerate this transition. Unlike static workflows, they can learn from context, reason through exceptions, and interact directly with both systems and people. This means AP teams are supported by autonomous assistants that not only process invoices, but also optimize working capital, monitor compliance continuously, and surface insights proactively.

Key Takeaways

• Cost savings: Mid-market firms can free up 200+ hours and save $180K–$300K annually.
• Compliance & accuracy: AI-driven automation reduces error rates by up to 80% and strengthens audit readiness.
• Future trends: Touchless AP, predictive analytics, AI-driven fraud detection, and finance-focused AI agents are moving from experimental to standard.
• Strategic growth: Invoice automation—powered increasingly by AI agents—is the bridge from back-office efficiency to finance-led decision-making.

Closing Thought: Invoice automation is no longer a “nice-to-have”—it’s an operational necessity. Companies that adopt AI-first platforms today position themselves not only to cut costs, but to build the finance function of the future. The next wave will be driven by AI agents—autonomous assistants that can handle exceptions, optimize cash flow, and proactively surface insights. The question isn’t if you should adopt automated invoice processing software, but how quickly you can put AI agents to work for your finance team.

Frequently Asked Questions about Invoice Automation

1. What is invoice automation and how does it differ from manual processing?

Invoice automation (or automated invoice processing software) uses AI to capture, validate, route, and pay invoices—cutting costs, speeding up cycle times, and reducing errors. Unlike manual processing, which relies on data entry and spreadsheets, automation provides touchless workflows that scale with your business.

2. How does AI-first invoice capture outperform traditional OCR?

AI-first capture doesn’t require rigid templates. It learns invoice patterns dynamically, adapts to layout changes, and maintains accuracy across thousands of vendor formats. Traditional OCR often fails when vendors update formats—leading to exceptions and manual fixes.

3. Can invoice automation handle multiple currencies and tax systems?

Yes. Most invoice automation solutions support multi-currency processing and local tax/VAT rules, making them effective for global operations. This ensures compliance and accuracy across jurisdictions while minimizing errors from manual entry.

4. What kind of time and cost ROI can mid-sized businesses expect?

For companies processing 1,000–2,000 invoices/month, automation can free up 200–400 staff hours monthly, cut costs from $15–20 per invoice down to ~$3, and unlock $180K–$300K in annual savings.

5. How long does implementation typically take?

Implementation depends on complexity and integrations, but most businesses go live in a few weeks to a few months. Many platforms include vendor support and pre-built connectors to accelerate rollout.

6. Will my team still need manual oversight after automating invoices?

Yes. Automation handles the majority of invoices, but exceptions—such as disputes, missing POs, or unusual spend—still require human review. This means AP teams spend less time on data entry and more time on strategy.

7. What size of business benefits most from invoice automation?

All business sizes benefit. Small firms gain efficiency and error reduction, mid-sized companies see the fastest ROI (200+ hours and six-figure savings annually), and large enterprises gain global compliance, scalability, and spend visibility.

8. How does automation improve vendor relationships?

By reducing delays and errors, automation ensures faster, more accurate payments. Supplier portals and better visibility improve communication, while timely payments strengthen trust and allow businesses to capture early-payment discounts.

Helsinki, Finland – Digital Workforce Services Plc continues to drive innovation in insurance claims processing with its Agent Workforce solution, now delivering measurable results for insurers and third-party administrators. Built on top of Microsoft Azure, the solution uses advanced agent capabilities and orchestration technologies from Microsoft Foundry, supported by open-source frameworks.

Since its launch, Agent Workforce has helped insurers reimagine their entire claims journey, from first notification of loss (FNOL) to final settlement, by deploying specialist AI agents for tasks such as data capture, coverage eligibility, fraud detection, adjudication, and settlement. The solution’s modular architecture enables rapid onboarding and integration with existing claims systems, accelerating transformation and improving customer outcomes.

Key Highlights:
• Proven Impact: Early adopters report significant reductions in manual processing time and improved accuracy in claims handling.
• Scalable Architecture: Powered by Azure, Agent Workforce offers enterprise-grade reliability, security, and scalability.
• Advanced Orchestration: Microsoft Foundry coordinates Agent reasoning workflows and orchestration.
• Industry Recognition: The solution is recognised for its ability to generalise across multiple insurance lines and adapt to evolving business needs.

“In traditional insurance operations, achieving more outcomes has always meant hiring more people. With Agent Workforce, powered by Microsoft Foundry and Microsoft Azure, we’re changing that equation. Our AI-native agents allow claims leaders to scale and improve their operations without increasing headcount. This means insurers can process more claims, deliver faster results, and realise unprecedented ROI, all while freeing skilled professionals to focus on the most complex and empathic work. Thanks to the flexibility and reliability of Microsoft technology, the Agent Workforce solution truly decouples human labour from outcomes, unlocking new levels of efficiency and growth,” said Karli Kalpala, Head of Strategy at Digital Workforce.

“Microsoft Foundry helps Agent Workforce deliver solutions that enable customers to focus on innovation and efficiency, while benefiting from the flexibility and enterprise-grade reliability that Microsoft Azure provides,” said Tarja Jernström, Commercial Partner Lead at Microsoft Finland. “The Agent Workforce solution addresses a variety of automation challenges and allows customers to take advantage of Azure’s advanced and future proof AI capabilities.”

Visit the officialAgent Workforce Product page

For more information
Karli Kalpala, Head of Strategy and AI Agent Business, Digital Workforce Services Plc, karli.kalpala@digitalworkforce.com

About Digital Workforce Services Plc
Digital Workforce Services Plc (Nasdaq First North: DWF) is a leader in business automation and technology solutions. With the Digital Workforce Outsmart platform and services—including Enterprise AI agents—organizations transform knowledge work, reduce costs, accelerate digitization, grow revenue, and improve customer experience. More than 200 large customers use our services to drive the transformation of work through automation and Agentic AI. Digital Workforce has particularly strong experience in healthcare, automating care pathways across clinical and administrative workflows to reduce burden, enhance patient safety, and return time to patient care. Following the acquisition of e18 Innovation, the company has further strengthened its position in the UK healthcare pathway automation. We focus on repeatable, outcome-based use cases, and we operate with high integrity and close customer collaboration. Founded in 2015, Digital Workforce employs more than 200 automation professionals in the US, UK, Ireland, and Northern and Central Europe. Our vision: Transforming Work – Beyond Productivity.
https://digitalworkforce.com | https://agent-workforce.com

The post Digital Workforce’s Agent Workforce Solution Gains Momentum on Microsoft Foundry and Microsoft Azure appeared first on Digital Workforce.

The post Top 10 Strategic Technology Trends for 2026 appeared first on AutomationEdge.

The post Top 10 GenAI Powered Employee Support Platforms for HR and IT Automation in 2026 appeared first on AutomationEdge.

The post AI and Automation Workflow Monitoring in 2026: The Definitive Guide to Maximizing Automation Success appeared first on AutomationEdge.

The post Automated Policy Administration for Better Operational Efficiency appeared first on AutomationEdge.

The post 9 Agentic AI Examples Showing Future of Intelligent Work appeared first on AutomationEdge.

The post Driving Financial Services Innovation : Harnessing the Power of Data, CRM, and AI appeared first on AutomationEdge.

Press release 14.1.2026, 8:00: North Savo and Pirkanmaa wellbeing services counties acquire Finland’s leading automation solution for archiving Uranus systems – total contract value €2 million

The wellbeing services counties of North Savo and Pirkanmaa have selected an advanced automation solution developed by Digital Workforce and Atostek to archive the retiring Uranus systems into Kela’s Kanta services. The solution is delivered to the wellbeing services counties as a SaaS service provided by Istekki. The combined value of the agreements is 2 M EUR. The same system-independent automation solution for extracting and transferring client and patient data has already been successfully deployed in over half of Finland’s wellbeing services counties.

The wellbeing services counties of North Savo and Pirkanmaa aim to decommission the remaining legacy Uranus systems within 18 months, shortly after the deployment of the new OMNI360 patient information systems. Through the timely decommissioning of legacy systems, the wellbeing services counties will realize significant cost savings, reduce the workload associated with maintaining multiple systems, and improve operational efficiency.

“At Pirkanmaa, the volume of data to be archived is substantial – covering more than three million personal identity codes. Deep partner expertise in the precise definition and management of the project is essential to ensure that the costs and workload associated with data transfer remain highly predictable. The purpose of the service solution we have procured is to ensure excellent control over the data migration project, consistent implementation quality, minimal workload for our own personnel, and the timely decommissioning of the retiring system. The archiving of the legacy system will begin early to avoid the long-term parallel operation of multiple systems. The annual cost of the Uranus system is significant for us, which is why the efficient decommissioning of the legacy system is critical as we transition to the new OMNI system”, says Juha Eerola, PTJ Project Manager at Pirkanmaa wellbeing services county.

“The archiving project for Uranus systems, which is now getting started, is significant in scope. The combined volume of data to be transferred by the wellbeing services counties covers approximately four million personal identity codes. Delivering the SaaS service efficiently requires substantial expertise and experience in both the technical automation solution and the specific requirements of Kanta archiving – capabilities that we can reliably provide through Atostek and Digital Workforce. There is still considerable work ahead across many wellbeing services counties: among Istekki’s customer-owners alone, hundreds of client and patient information systems scheduled for decommissioning remain unarchived” says Ari-Pekka Häyrynen, Business Manager at Istekki.

“Together with Atostek, we have delivered a higher number of automation-based Kanta archiving projects and client and patient data migrations in Finland than any other provider. The results of these projects have been excellent regardless of the systems involved. We consider it essential that the remaining archiving and data transfer projects are carried out cost-effectively, on schedule, and with high quality. Once the burden of maintaining legacy systems is removed, wellbeing services counties can focus on developing their operations and improving productivity, says Juha Nieminen, Head of Healthcare Nordics at Digital Workforce.

“Our Kanta archiving solution is built on Atostek’s ERA platform – the most extensive information system in Finland utilizing Kanta services – combined with Digital Workforce’s robotic process automation (RPA). Data extraction is carried out using RPA, enabling an agile, system-independent implementation. Data conversion and transfer to the Kanta archive are then executed through the ERA platform, which has been specifically developed for social and healthcare information management”, explains Miika Parvio, Business Director of ERA Services at Atostek.

For more information:
Marja Heikkinen, Key Account Manager, Digital Workforce
Email: marja.heikkinen@digitalworkforce.com

About Digital Workforce Services Plc

Press release 14.1.2026, 8:00: North Savo and Pirkanmaa wellbeing services counties acquire Finland’s leading automation solution for archiving Uranus systems – total contract value €2 million

The post North Savo and Pirkanmaa wellbeing services counties acquire Finland’s leading automation solution for archiving Uranus systems – total contract value €2 million appeared first on Digital Workforce.

Digital Workforce is refining the company’s business structure. Going forward, the business will be managed through two global business areas: Healthcare and Enterprise & Public. Shared global functions will ensure operational efficiency and scalability. Financial information will continue to be reported at the company level.

The following appointments have been made in the company’s management team:

Juha Nieminen has been appointed as Chief Growth Officer of the Healthcare business area. Juha previously led the company’s healthcare business in the Nordics, as a member of the management team.

Tapio Niinikoski (M.Sc., Tech) has been appointed as Chief Growth Officer of the Enterprise & Public business area and as a member of the management team. Tapio joins the company from outside and will start in his position on 2 February 2026. He has strong experience in business process automation for international large enterprises and the public sector. He has leadership experience as both sales and country director in leading digital business companies such as Digia, Basware and Elisa.

In addition, the following changes will be made in the management team:

Karri Lehtonen (Head of Sales, North America and Head of Legal) and Kristiina Åberg (Head of Marketing) will continue in their current roles but will step down from the management team.

Additionally, Stefan Meller, who has been responsible for Europe region sales to the Enterprise & Public customers, will take on responsibility for business area accounts and continue in the company but will step down from the management team.

Jussi Vasama, CEO of Digital Workforce:

“I would like to thank all members of the management team for their valuable, systematic, and long-term work in developing the company into a leading player in business process automation and a frontrunner in the industry. Going forward, our strategic focus will continue to be on operational concentration, strengthening industry knowledge, and scalable service offerings to accelerate profitable growth.

I am very pleased to welcome Tapio Niinikoski to our management team. Tapio is an experienced sales and business leader who has demonstrated strong performance in demanding international environments. His approach, built on collaboration and trusted customer relationships, fits our company well as we move into the next phase of our development.”

As of 2 February 2026, the management team of Digital Workforce services Plc will consist of the following members:

– Jussi Vasama (CEO)
– Laura Viita (CFO)
– Mikko Lampi (COO)
– Juha Nieminen (Chief Growth Officer, Healthcare)
– Tapio Niinikoski (Chief Growth Officer, Enterprise and Public)
– Karli Kalpala (Head of Strategy & Agentic AI business)
– Louise Wall (Managing Director, Healthcare UK and Ireland)
– Eila Onniselkä (Head of People & Culture)

Contact information:
Digital Workforce Services Plc
Jussi Vasama, CEO
Tel. +358 50 380 9893

Laura Viita, CFO
Tel. +358 50 487 1044
Investor relations | Digital Workforce

Certified advisor 
Aktia Alexander Corporate Finance Oy
Tel. +358 50 520 4098

The post Changes in Digital Workforce Services Plc.’s business areas and management team – Juha Nieminen and Tapio Niinikoski appointed as Chief Growth Officers of business areas appeared first on Digital Workforce.

We are excited to meet fellow attendees, exhibitors, and speakers to share ideas and build on each other’s successes in advancing impactful and productive healthcare.

Come meet us to discuss how automation and AI help build and manage diverse care pathways, streamline administrative work, and ensure systems truly support human needs — for both healthcare professionals and their patients.

Our team is happy to share insights and real-world results from some of the largest hospitals and healthcare organizations across the Nordics, the UK, and the US.

See you there!

Event details & tickets ->

Radical Health Festival is co-curated with the Finnish Government—through the Ministry of Social Affairs and Health and the City of Helsinki—bringing together bold thinkers and doers from across healthcare, policy, technology, research, and civil society.

The event will take place at Messukeskus, Helsinki Expo and Convention Centre, on 19–21 January 2026.

Schedule:

Monday, 19 January
Welcome Reception | 18:30–19:30

Tuesday, 20 January
Festival Programme | 08:15–18:00

Wednesday, 21 January
Festival Programme | 08:30–17:00

Tickets: 
radicalhealthfestival.com

The post Digital Workforce team joins Radical Health Festival in Helsinki appeared first on Digital Workforce.

Digital Workforce Services Plc has signed an agreement with a leading Nordic enterprise to modernize finance and payroll operations following a large-scale merger. The agentic solution is built on UiPath’s orchestration platform and will help streamline month-end closing and payroll data processing across multiple business units.

The project brings together several core systems—including client management, financial software, support ticketing, and payroll platforms—into a single, orchestrated workflow. Digital Workforce will deliver an end-to-end workflow solution that combines AI-driven agents, orchestration, automation, and human review to ensure critical steps remain governed and auditable while forming a seamless solution that leverages the customer’s existing process automation.

A key factor in securing the agreement was DWF’s enterprise-grade delivery model. This included discovery workshops, architectural assessments, and a secure deployment within the customer’s Microsoft Azure environment. The solution is designed to align with enterprise IT requirements while protecting sensitive finance and HR data and maintaining full traceability of actions and decisions.

“This project shows what agentic automation can achieve in complex, regulated environments,” said Jussi Vasama, CEO, Digital Workforce Services Plc. “By combining the speed of AI and automation with human judgment and oversight, we can help customers move faster and improve accuracy, without compromising control or compliance.”

Vasama added: “For Digital Workforce, this is a significant milestone. It confirms our ability to deliver orchestrated, agentic solutions at enterprise scale for large organizations, and strengthens our role as a trusted transformation partner in the Nordics. It also accelerates our strategy to deliver measurable business outcomes to enterprise customers through governed automation.

For more information
Jussi Vasama, CEO, Digital Workforce Services Plc, jussi.vasama@digitalworkforce.com

About Digital Workforce Services Plc
Digital Workforce Services Plc (Nasdaq First North: DWF) is a leader in business automation and technology solutions. With the Digital Workforce Outsmart platform and services—including Enterprise AI agents—organizations transform knowledge work, reduce costs, accelerate digitization, grow revenue, and improve customer experience. More than 200 large customers use our services to drive the transformation of work through automation and Agentic AI. Digital Workforce has particularly strong experience in healthcare, automating care pathways across clinical and administrative workflows to reduce burden, enhance patient safety, and return time to patient care. Following the acquisition of e18 Innovation, the company has further strengthened its position in the UK healthcare pathway automation. We focus on repeatable, outcome-based use cases, and we operate with high integrity and close customer collaboration. Founded in 2015, Digital Workforce employs more than 200 automation professionals in the US, UK, Ireland, and Northern and Central Europe. Our vision: Transforming Work – Beyond Productivity. https://digitalworkforce.com | https://agent-workforce.com

The post Digital Workforce to Deliver Transformative Agentic Automation Solution for Leading Nordic Enterprise appeared first on Digital Workforce.

Introduction: When Your Automation Breaks Down

Imagine you've trained the perfect robotic assistant to handle your company's invoice processing. It reliably extracts numbers from specific invoice formats, follows your approval rules to the letter, and sends payments exactly on schedule. Then one day it receives an invoice where the vendor’s name is slightly different, the amount requires special approval, and the formatting doesn't match anything in its rules. Your perfect automation grinds to a halt, requiring human intervention.

This is the fundamental limitation of traditional Robotic Process Automation (RPA). It excels at deterministic, rule-based tasks—the "if this, then that" processes that computers have been handling for decades. But in a business world filled with unstructured data, exceptions, and constantly changing requirements, these systems often create as much maintenance work as they save.

The Three Pillars of Intelligent Automation

The next evolution of automation combines three powerful technologies to create systems that don't just follow instructions but understand context, learn from experience, and make reasoned decisions.

1. The Understanding Layer: Large Language Models (LLMs)

Large Language Models (LLMs)—the technology behind tools like ChatGPT—act as the "brain" of intelligent automation. Unlike traditional software that requires explicit programming for every scenario, LLMs can:

• Interpret unstructured information like emails, documents, or chat messages
• Understand intent even when expressed in different ways
• Make judgment calls based on guidelines rather than rigid rules
• Explain their reasoning in human-understandable terms

For example, where traditional automation might fail with an invoice in a new format, an LLM-powered system can read and understand the document much like a human would, extracting relevant information even if it's presented differently than expected.

2. The Memory Layer: Graph-Based Knowledge Systems

If LLMs provide understanding, graph-based systems provide contextual memory. Traditional databases store information in tables (like spreadsheets), but graph databases store information as interconnected nodes and relationships.

Think of it this way: A traditional database might tell you that "Invoice #456 is for $5,000." A graph database tells you that "Invoice #456 is FROM Vendor-A, who HAS_A_CONTRACT with us that ALLOWS purchases up to $10,000 without special approval, and this invoice is FOR Project-X, which IS_MANAGED_BY Jane Doe."

This interconnected "memory" allows automated systems to:

• Recall past decisions and their outcomes
• Understand relationships between different pieces of information
• Make decisions based on rich context rather than isolated data points

3. The Action Layer: The Agent Orchestrator

The agent orchestrator is the practical manager that ties everything together. It receives a task or goal, breaks it down into steps, decides which tools to use, executes the plan, and learns from the results. It's like a project manager coordinating between the "understanding" (LLM) and "memory" (graph database) to get things done.

Here's a simplified view of how these components work together:

Real-World Applications: Where Intelligent Automation Excels

Adaptive Customer Service

Traditional chatbots follow decision trees—if a customer says "A," the bot responds with "B." Intelligent agents can understand a customer's actual problem, pull up their entire history with your company, review relevant policies, and provide personalized solutions. If a solution requires multiple steps (issuing a refund, sending a replacement, updating account notes), the agent can handle the entire process seamlessly.

Context-Aware Document Processing

Consider an insurance claim that includes medical reports, police documentation, and photos. An intelligent system can:

1. Understand and extract relevant information from each document type
2. Cross-reference this claim with similar past claims in its memory
3. Check for consistency and potential red flags
4. Route the claim to the appropriate adjuster with a summary and recommended actions

Dynamic Project Management

Instead of simply tracking tasks and deadlines, an intelligent project management assistant could:

• Understand the dependencies between different project components
• Anticipate potential bottlenecks based on similar past projects
• Proactively suggest resource reallocations when delays occur
• Generate progress reports tailored to different stakeholders' needs

Building Your First Intelligent Agent: A Practical Framework

Step 1: Start with a Contained Use Case

Don't try to automate your most complex process first. Choose something manageable with clear boundaries, like:

• Sorting and categorizing internal support requests
• Processing a specific type of standardized form
• Gathering and summarizing daily reports from multiple sources

Step 2: Design Your Knowledge Graph

Before building anything, map out what your system needs to "know." Identify:

• Key entities (people, projects, documents, products)
• Important relationships (approves, manages, contains, depends on)
• Critical attributes (status, priority, deadline, amount)

Step 3: Implement with Guardrails

Intelligent systems need boundaries. Establish clear rules about:

• What decisions the system can make autonomously vs. what requires human approval
• How confident the system needs to be before acting
• Where to log all decisions and actions for review and improvement

Step 4: Adopt an Iterative Approach

Start with the system in an "assistive" role, suggesting actions for human review. Gradually expand its autonomy as you gain confidence in its performance. Continuously add to its knowledge graph based on real-world outcomes.

The Human Advantage: Collaboration, Not Replacement

The most successful implementations of intelligent automation view these systems as collaborative partners rather than replacements for human workers. The technology handles repetitive cognitive work, exception handling, and information synthesis, freeing humans to focus on:

• Strategic decision-making based on the synthesized information
• Relationship management that requires emotional intelligence
• Creative problem-solving for truly novel challenges
• Oversight and governance of the automated systems

In accounting departments, for instance, intelligent agents might handle 80% of routine invoice processing, flagging only the complex exceptions for human review. The accountants then spend their time on analytical work, process improvement, and managing vendor relationships—higher-value activities that leverage their expertise.

The Road Ahead: Where This Technology is Heading

As intelligent automation evolves, we're moving toward systems that can:

1. Learn continuously from every interaction without explicit reprogramming
2. Explain their reasoning transparently, building trust with human colleagues
3. Collaborate with each other, with different specialized agents working together on complex processes
4. Adapt proactively to changing business conditions and requirements

This represents a fundamental shift from viewing automation as a way to reduce headcount to viewing it as a way to augment organizational intelligence—creating enterprises that are more resilient, adaptive, and capable of handling complexity.

Getting Started with Intelligent Automation

If you're considering implementing intelligent automation in your organization:

1. Audit your processes to identify candidates with:
• High volumes of unstructured data (emails, documents)
• Many exceptions or variations
• Requirements for contextual understanding
2. Start small and focused with a pilot project that has clear success metrics
3. Invest in knowledge organization—the quality of your graph structure will determine the quality of your automation
4. Plan for change management—help your team transition from overseeing rules to managing intelligence

The most forward-thinking organizations aren't asking "Which tasks can we automate?" but rather "How can we create systems that understand what we're trying to accomplish and help us do it better?"

This article builds on concepts explored in other pieces on AI Quantum Intelligence, including our looks at why traditional RPA often fails without AI, practical applications of automation using AI, and methods for evaluating multi-step AI-generated content.

What's the first process in your organization that could benefit from understanding, not just following rules? Share your thoughts in the comments below.

Written/published by Kevin Marshall with the help of AI models (AI Quantum Intelligence).

The post Banking Technology Trends: Forecasting the Future of Finance in 2026-27 appeared first on AutomationEdge.

The post Automation Using AI: 5 Game-Changing Examples You Can Implement Today appeared first on AutomationEdge.