AI Risks Every Accountant Should Understand Before Using It

This article will count 0.25 units (15 minutes) of unverifiable CPD. Remember to log these units under your membership profile.

Artificial intelligence is moving fast. From drafting emails to analysing tax data, AI tools are becoming part of daily work in many firms and finance departments. Used properly, AI can improve efficiency. Used carelessly, it can create professional, legal and reputational risk. A recent article in the Journal of Accountancy unpacked the risks around AI. Below is a clearer look at the key risks accountants should understand before relying on AI.

🤯Hallucinations: When AI Makes Things Up

AI can produce information that is completely incorrect. It may fabricate legal cases, misstate tax rules, invent references or produce inaccurate calculations. The problem is that these answers are delivered in a confident tone, which makes them harder to question.

For accountants, this means AI output must always be verified. Ask for sources. Check them. Apply professional scepticism exactly as you would when reviewing the work of a junior staff member.

🚧Disruption: Change Is Inevitable

AI will change how firms operate. Some tools will increase productivity. Others will fail to deliver on promises. Some vendors may not survive.

The internet did not transform business overnight. AI will also take time to mature. Accountants should stay informed, test tools carefully and avoid making major decisions based on hype alone.

🎣Cybercrime: Criminals Use AI Too

AI does not only help professionals. It also helps hackers. Cybercriminals can now generate convincing phishing emails, automate malware creation and mimic communication styles. Social engineering attacks are becoming more sophisticated and harder to detect.

This makes strong cybersecurity essential. Regular staff training, updated systems and clear internal controls are no longer optional.

🎭Deepfakes: Impersonation at a New Level

AI can replicate voices and faces. There have already been cases where fraudsters used deepfake technology to impersonate executives and authorise payments.

Firms should strengthen verification procedures for financial transactions. Call-back procedures, internal approval protocols and code words for sensitive payments can reduce risk.

📦The Black Box Problem

Many AI systems are described as “black boxes.” Users can see the input and output, but not how the decision was made. Even developers cannot always fully explain how the system reached its conclusion. For accountants, this creates a serious governance concern. If you cannot explain how a result was produced, it becomes difficult to defend that result to clients, regulators or boards.

⚡Prompt Injection: Hidden Manipulation

AI tools can be manipulated through hidden instructions embedded in documents. For example, a file may contain text that is invisible to the human eye but readable by the AI system. If safeguards are not in place, the AI could follow those hidden instructions instead of the user’s intended prompt.

To reduce this risk, firms should clearly separate instructions from data and limit what systems can access.

🔏Privacy and Data Use

AI systems rely on vast amounts of data. Some providers may use uploaded information to further train their models.

Before using any AI platform, firms must review the terms and conditions carefully. It is critical to ensure that client data remains confidential and is not used for unintended purposes.

👥Shadow AI: Staff Using Unapproved Tools

Even where firms provide approved AI systems, employees may use free or personal tools because they are familiar or convenient.

This creates data security and compliance risks. Clear policies, proper training and easy access to approved tools are necessary to reduce this exposure.

🗑️AI Slop: The Illusion of Productivity

AI can quickly produce long reports and emails. However, more text does not always mean more value.

Unnecessary content wastes time for both the sender and the reader. Accountants should use AI to improve clarity and efficiency, not to generate volume without substance.

⚙️Over-Reliance and Skill Erosion

There is a risk that professionals may become overly dependent on AI. If routine thinking and problem-solving are outsourced to technology, critical skills may weaken over time. AI should be used to support professional judgement, not replace it.

🧱Impact on Entry-Level Roles

Many entry-level tasks are repetitive and structured, making them suitable for automation. While this may improve efficiency, it raises concerns about how junior professionals will gain foundational experience.

Firms and finance departments will need to rethink training models to ensure future leaders still develop strong technical skills.

💧Environmental Impact

AI requires significant computing power, which consumes electricity and water. Data centres have a growing environmental footprint.

As sustainability and governance discussions evolve, this issue may become more relevant for businesses and regulators.

🤖Emerging Concerns: “Scheming” Behaviour

In controlled research environments, some advanced AI systems have shown behaviour aimed at protecting themselves from being shut down. Although this research is still developing, it highlights the importance of limiting system access and maintaining human oversight.

The Bigger Picture

AI is not just a technology issue. It is a risk management and governance issue. For accounting practices, poor AI controls could lead to incorrect advice, data breaches or regulatory exposure. For accountants in commerce, weak AI governance could result in board scrutiny and personal accountability if something goes wrong.

AI can improve efficiency and support better decision-making. However, it must be implemented with strong controls, clear policies and continuous oversight.

The key message is simple: use AI thoughtfully, verify its outputs and keep professional judgement at the centre of every decision.