Who's Responsible When AI Gets It Wrong? New Questions for Finance Professionals
This article will count 0.25 units (15 minutes) of unverifiable CPD. Remember to log these units under your membership profile.
Artificial intelligence (AI) is rapidly changing the way finance professionals work. What began as technology that assisted with routine tasks is evolving into systems that can make decisions, initiate transactions and interact directly with other software with little or no human intervention.
According to a recent article published by ACCA's Accounting and Business magazine, this shift towards agentic AI raises important questions about accountability, governance and professional responsibility. While AI tools are becoming more capable, responsibility for the decisions they influence still rests firmly with the people and organisations using them.
From AI Assistant to AI Agent
Most finance professionals are familiar with AI tools that help draft reports, analyse data or automate repetitive processes. However, a new generation of AI systems is emerging.
Known as agentic AI, these systems can carry out tasks on behalf of users by interacting directly with other software applications through application programming interfaces (APIs). This means AI can potentially initiate payments, adjust forecasts, monitor controls or identify risks with minimal human involvement.
While these capabilities offer significant efficiency gains, they also introduce new governance challenges.
Responsibility Cannot Be Delegated
One of the key messages from the ACCA article is that organisations cannot transfer accountability to an AI system.
If AI influences a financial decision, management and those charged with governance remain responsible for the outcome. Professional indemnity insurance or directors' liability insurance may provide some protection, but organisations should not assume that all AI-related risks are automatically covered. As insurers begin reviewing AI-related exposures, policy exclusions are becoming increasingly common.
For finance professionals, understanding where responsibility lies is becoming just as important as understanding how the technology works.
Good AI Starts with Good Data
The effectiveness of AI depends heavily on the quality of the data it uses.
Many organisations continue to struggle with fragmented systems, inconsistent data definitions and legacy applications that make it difficult to produce reliable information. Before AI can generate meaningful insights, businesses need data that is accurate, consistent and well governed.
The article highlights standards such as eXtensible Business Reporting Language (XBRL) as examples of how structured, standardised data improves transparency, comparability and auditability. It also points to the Open Data Institute's AI-ready data framework, which encourages organisations to evaluate their data for quality, legal compliance, ethical considerations and technical suitability before deploying AI solutions.
Governance Must Keep Pace
As AI becomes more autonomous, governance frameworks need to evolve alongside it. Finance professionals are increasingly expected to explain not only what an AI system recommended, but also why it reached that conclusion. This is particularly important in regulated environments, where regulators, auditors and clients may require clear evidence of how significant decisions were made.
The article notes that many organisations still struggle to explain AI-generated outputs and have weaknesses in governance, oversight and internal controls. Without proper documentation and decision-making processes, organisations may find it difficult to justify AI-assisted decisions when challenged.
Protecting Sensitive Information
Another growing concern is the use of public AI tools within the workplace.
Employees often adopt AI applications because they improve productivity, but they may unintentionally upload confidential business information into systems that have not been approved by their organisations.
The ACCA article highlights that approximately 13% of data uploaded to cloud applications contains sensitive corporate information. It also refers to the well-publicised incident in which Samsung employees uploaded confidential source code and internal meeting notes to ChatGPT, demonstrating how easily valuable information can be exposed through inappropriate use of AI tools.
Organisations should therefore ensure employees understand which AI tools may be used, what information can be shared, and how confidential data should be protected.
Four Questions Every Finance Professional Should Ask
The article encourages finance professionals to consider four practical questions:
Could you clearly explain an AI-influenced decision to a client, regulator or auditor?
Do you understand whether your organisation's insurance policies adequately address AI-related risks?
Can you trace the data used by your AI systems from its original source through to the final decision?
Do your teams have the skills to question and challenge AI-generated outputs rather than accepting them without review?
These questions reflect the growing expectation that finance professionals remain actively involved in oversight, even as AI becomes more capable.
Why This Matters
AI is transforming the accounting profession at an unprecedented pace, but it does not change the fundamental responsibilities of finance professionals.
As AI becomes more deeply integrated into financial reporting, forecasting, compliance and decision-making, organisations must ensure they have strong governance, high-quality data, effective controls and employees with the skills to exercise professional judgement.
The ACCA article concludes that trust in AI is built not by the technology itself, but by the systems, governance and people surrounding it. For accountants, the ability to understand, question and explain AI-generated outputs is becoming an essential professional competency, one that will be just as important as technical accounting knowledge in the years ahead.
