780,000 Cases, 42,500 Hours: What the DWP's AI Eligibility Pilot Actually Measured

Will Drewes

Founder, Fern Strategy · 3 min read

One agency published the numbers. Most haven't.

The UK Department for Work and Pensions ran an AI-assisted eligibility workflow on Employment Support Allowance claims and disclosed the results publicly. That alone makes it worth examining. Most agencies piloting similar tools have not published anything this specific.

The headline figures: over 780,000 cases processed, with 42,500 operational hours saved. The system cross-references applicant health conditions against claim criteria and surfaces a recommendation. A human caseworker makes the final decision.

That human-in-the-loop structure matters, and the accuracy history explains why it's still there.

The accuracy gap is the real story

The DWP disclosed that the current system runs at 87% prediction accuracy. That sounds reasonable until you see what came before it. An earlier version, used from 2020 to 2024, hit only 35% accuracy. Caseworkers had to manually intervene on the remaining 65% of cases.

Think about what that earlier version actually cost in practice. If you're deploying a tool that's wrong nearly two-thirds of the time, you haven't reduced caseworker burden. You've added a review step on top of the original workload. The 87% figure represents a real operational improvement, but it also tells you the first four years were spent learning what didn't work.

The DWP's current design acknowledges this honestly: the system assists, it doesn't decide. That's not a hedge. Given the track record, it's the correct architecture.

What the research doesn't show yet

Beyond the DWP data, the public record on AI in government benefits is thin on hard numbers. Code for America has piloted tools to help workers answer policy questions and review eligibility documents for SNAP, but no outcome metrics have been published yet. Separate research testing LLMs against state-level SNAP and Medicaid rules found accuracy rates of 0% on state-specific numerical thresholds in some cases. Those tools were not deployed, but the finding is a useful reminder: a model that handles federal policy text can still fail completely on a state's specific dollar limits or household size tables.

The gap between "works in a demo" and "works on your actual ruleset" is where most of these projects stall.

What this means for benefits administrators and their oversight teams

The DWP case gives you a concrete benchmark and a realistic timeline. Four years to get from 35% to 87% accuracy is not a failure story, but it is an honest one. Any agency evaluating AI for eligibility processing should ask three questions before committing budget:

1. What is the baseline accuracy on your specific rules, not a generic benchmark? State-level variation in benefits policy is significant. A system trained on federal guidelines may not handle your edge cases.
2. Where does the human decision boundary sit, and is it auditable? The DWP's design keeps the final call with a caseworker. The audit trail for why a recommendation was made needs to be readable by a reviewer, not just a data scientist.
3. What does the error mode look like? At 65% error rate, the DWP's earlier system created more work than it saved. Knowing how a system fails before you deploy it at scale is the difference between a pilot and a problem.

The 42,500 hours saved is a real number. So is the four-year accuracy climb that preceded it. Both belong in your business case.