There’s a story accountants are being told about AI. It goes like this: feed the software your tax manuals, HMRC guidance, FRS rules, and it absorbs them like a vsponge. Ask it a compliance question, and it reaches into that knowledge and gives you the right answer.
It’s a compelling story.
It’s also wrong.
And that gap between the story and reality is where the profession’s AI risk actually lives.
The Mental Model Most Vendors Are Selling You
If you tune into almost any AI demo right now you’ll see an impressive piece of software, but here’s what most vendors won’t tell you.
A large language model doesn’t read rules and apply them. It predicts the most probable next word, based on patterns in its training data. It’s a sophisticated pattern matcher. It’s not a reasoning engine or a rulebook reader.
A model trained on the open internet (Reddit threads, outdated practice forums, contradictory commentary) will produce answers that look correct with considerable confidence. In most industries, ‘probably correct’ is good enough, but not in accounting. In this regulated profession, where the answer is either right or wrong and the liability falls on the practitioner, probably correct is a professional risk.
This shouldn’t be a reason to avoid AI, but it should justify the demand for AI in this industry to be built differently.
Three Types of AI Vendors
The current AI market breaks into three categories, and being able to distinguish them is the first step in any sensible due diligence.
- Generic copilots: This is essentially ChatGPT integrations bolted onto existing products. It’s useful for drafting emails and summarising documents, but they don’t know what a 64-8 is. The compliance depth simply isn’t there.
- Workflow automation: This is rule-based logic presented as intelligence. It’s good for routing tasks and sending reminders, but it can’t reason about specific regulatory outcomes. The gap between what it’s marketed to do and what it can actually do is wide.
- Purpose-built AI: This is AI designed from the ground up for the specific jobs accounting practices need to do. Knowledge is constrained to authoritative sources and human oversight is at every regulated decision point.
Only the third category deserves a serious conversation for mandated compliance work.
The Six Questions That Cut Through the Noise
Before you sign anything, you need six things answered (clearly, with evidence, in writing).
- Where does your data actually go?
You remain the data controller regardless of what your vendor does behind the scenes. If the answer is “we use ChatGPT” without written commitments on data routing, geographic residency, and zero-training exclusions, that conversation should end immediately. - Where does the AI’s knowledge come from?
Is it grounded in legislation from gov.uk and HMRC, or trained on general internet text? The difference is the difference between a tool you can trust and one that occasionally produces confident-sounding errors. For example, an AI whose training predates April’s MTD IT launch date isn’t fit for purpose, full stop. - Can they prove their accuracy mathematically?
“Trust us, it works” is a marketing slogan, not security. Benchmark results against verified datasets are evidence. Ask for them. - When it breaks, can they replay exactly what happened?
HMRC enquiries, PI claims, and client disputes all require you to replay what was done, when, on what basis. Every AI action needs a full audit trail: timestamp, model version, input, output, confidence score. If they can’t replay a transaction from six months ago, you’re buying a black box. - What models are actually under the hood?
The right architecture for compliance work is multi-model routing. A vendor using one massive general-purpose model for every compliance job is not optimal. - Is this a living system or a launched product?
Legislation changes. An AI that was accurate at launch may be materially out of date within months. Ask how fast the system ingests new legislation, and if qualified chartered accountants actively involved in validating changes before they reach customers.
The Standard the Market Should Be Held To
The vendors making it hardest to get clear answers to these questions are the ones with most to hide. Credible AI built for this profession can answer every one of them plainly, with evidence, and put the commitments in writing.
The accounting profession is at a genuine inflection point. AI done well transforms practice capacity. It reduces non-billable admin, handles client chasing, accelerates onboarding, and frees you up for advisory work that actually grows the practice.
But AI done wrong doesn’t just underperform. It creates professional exposure that your PI cover may not protect you from.
The AI rulebook myth is costing this profession. Our guide, “Six Questions Every Accountant Should Ask Before Buying AI”, sets out the full due diligence framework, including how Bright answers each question in writing.
