What machine learning really does, and why you still make the call

This article will count 0.25 units (15 minutes) of unverifiable CPD. Remember to log these units under your membership profile.


A supplier invoice for R96,300 lands in the payables run. The supplier usually bills about an eighth of that, its last invoice came in two days ago, and the banking details are new. In a batch of 2,000 invoices on a busy Friday, you might miss it. A well-trained machine learning model would not, and it would flag it for you to check.

That is what machine learning does well: it finds what looks out of place in far more data than any person could work through. A flag is not a finding, though. The invoice may be a genuine one-off order, and only a person can find out what happened and decide what to do next.

Rules versus patterns

Most accounting software follows fixed rules. “Flag every invoice over R100,000” is a rule, and changing the limit to R200,000 means someone has to change the setting. Machine learning works from examples instead. Show it thousands of past invoices and it learns what normal looks like for each supplier, so the instruction becomes “show me anything unusual”.

For our invoice, the model looks at the amount compared with the supplier’s average, how recently the supplier last billed, whether the bank account is new and the supplier’s history. It then gives the invoice an anomaly score. A routine invoice might score 8 out of 100, while this one scores 92. That is not a 92% chance of fraud. It means the invoice is very different from anything the model has seen from this supplier, which is your cue to look.

A 70-year-old idea

ChatGPT made AI feel new in November 2022, but the idea behind it is more than 70 years old. In 1950 the British mathematician Alan Turing asked whether a machine could think, and suggested that if you could not tell a machine’s replies from a person’s, you might as well call it intelligent. Six years later, a group of scientists at Dartmouth College in the United States gave the field its name: artificial intelligence.

The early systems ran on hand-written rules. ELIZA, a chatbot built in the mid-1960s, imitated a therapist by spotting keywords and turning your sentence back into a question. It worked up to a point, but these systems could not cope with situations their programmers had not foreseen, and with little data and weak computers, progress stalled and funding dried up for years.

Three things changed that after 2000: huge amounts of data, cheap computing power through the cloud, and better algorithms that let computers learn patterns from data instead of following a rule for every situation. The milestones followed. IBM’s Deep Blue beat the world chess champion in 1997, IBM’s Watson won the quiz show Jeopardy! in 2011, and in 2016 Google DeepMind’s AlphaGo beat one of the world’s best players of Go, a game with too many possible moves to program by hand. In 2017 a new design called the transformer made large language models possible, and five years later ChatGPT let anyone use them by typing a question in plain language.

Poor data in, poor answers out

A model is only as good as the examples it learned from. If an insurer trains a model only on declined claims, it has no idea what a valid claim looks like, and every new claim starts to look suspicious. That is how bias creeps in.

The same applies in a finance team. Say a business has always paid its suppliers within 30 days but has just negotiated 60-day terms. A cash flow model built on the old pattern will keep forecasting the wrong outflows until someone gives it the new terms. This gap between what the model learned and what is happening now is called model drift, and it is why models need to be checked and retrained.

Where you will meet it

You may already be using machine learning without calling it that, because Sage and many ERP systems now build it in for analysis and exception reporting alongside the normal accounting functions. The most common uses in finance are:

  • Catching unusual transactions: large amounts, odd timing, new bank details and likely duplicates, such as two invoices with different reference numbers but the same amount, date and description.

  • Forecasting: projecting cash flow from past receipts, seasonal patterns and customer habits, for example a customer who always pays 15 days late. If that customer has just renegotiated its terms, the forecast needs adjusting.

  • Audit and review: checking every transaction instead of a sample, and highlighting the few that need a closer look. Running a model over the whole population does not on its own give you sufficient appropriate audit evidence, so your follow-up procedures still depend on the engagement and its risks.

AI agents go a step further. Give an agent a goal, such as “reconcile the March supplier statements and flag any differences”, and it will fetch the statements, match invoices against your records and draft queries to suppliers. Set up well, it stops there and waits for you to approve. The risk comes when an agent can act on its own, because one that can change supplier banking details or start a payment can do real damage if it misreads an instruction. Put access limits, approval steps and transaction limits in place before you connect an agent to any financial system.

What this means for you

A model predicts; it does not understand. It may flag the R96,300 invoice without knowing that the business placed an unusually large order last week, or forecast cash flow without knowing about a customer agreement signed yesterday, new tax legislation or a change in interest rates.

That changes the accountant’s role. You do not need to build the models, but you do need to understand what they are for, question what they produce and add the context they lack. For the R96,300 invoice, that means checking the supplier’s records, confirming the new banking details independently and making sure the order was authorised before anything is paid. The technology shows you where to look, and the decision, along with the responsibility for it, stays with you.

AW covered a related risk in Your AI tool never says “I’m not sure”.

Practical tips: six questions to ask before you trust the output

  1. Is the data accurate, relevant and protected? Poor data gives poor predictions, and client information must stay within your confidentiality and POPIA obligations.

  2. What is the system designed to do? Find out whether it flags anomalies, forecasts, classifies transactions or takes actions. A tool that only flags carries far less risk than one that can pay, post or change records.

  3. How reliable is the result? Ask your software provider how the model was tested, what its known limitations are and whether it is monitored and retrained as conditions change.

  4. What evidence supports the output? Go back to the invoice, contract or bank record behind every flag. A high score on its own does not prove a transaction is wrong.

  5. What has changed since the model learned its patterns? Think about new legislation, interest rates, revised payment terms and changes in a client’s circumstances, and whether the model knows about them.

  6. Who is accountable for the final decision? Set clear approval points and access limits, and get your client’s authorisation before any agent touches their bank or accounting records.

Let the technology do the first pass, and check the evidence before you act on what it tells you.

Want to learn more?

This article is based on CIBA’s webinar Understanding machine learning: what it really does and why it matters, held on 27 October 2026. The one-hour session explains in plain terms how machine learning systems learn from data, what they depend on and where they fall short, and it carries 2 CPD units.



 

Trending


Latest Podcast



Next
Next

Can You Trust What AI Helped Build?