Checking AI

AI can make mistakes

No one’s perfect – and that applies to AI too. These tools can sometimes get things wrong. Let’s look at some of the ways this can happen.

AI bias 

AI is designed and trained by people using data created by people. It can sometimes reflect their biases. So it can work better for some people than others. Bear this in mind especially when you’re looking at AI-generated news items or if you’re using AI to help you create content – whether that’s help with writing a formal letter, creating a post on social media or drafting an email. 

Confident but incorrect

Think about predictive text on your phone. As you type, it suggests the next word. That’s AI at work. Based on what it’s seen before, it works out what word is most likely to come next. But AI doesn’t know the context – who you’re messaging or what you’re thinking. It’s making an educated guess. So if you type ‘See you at the…’ it might suggest ‘office’. But what you want to say is ‘zoo’. That’s a less common word, so AI is less likely to predict it.

So, AI sometimes mistakes ‘what sounds likely’ for ‘what is actually true’. This is called ‘hallucination’. It happens because AI uses maths, patterns and probability to work out the answer. It’s why AI may give us information that’s incorrect, misleading, or completely made up. Sometimes, the information isn’t completely wrong, but it just doesn’t make sense in the context you intended. AI can be overconfident in its answers, too.

In particular, be very wary of AI suggestions around health diagnoses and treatment options.

Mistakes can include:

Check before you trust

It’s good to get into the habit of checking answers or suggestions that AI gives you. As we’d said earlier, it doesn’t always get things right. What, when and how much you check will depend on what you’re asking and how you’ll use the information. 

What to check

When to check

Examples where you need to check:

• Research – You’ll need to cross-check with trusted sources and check any quoted sources exist and are up-to-date.
• Health information – AI is OK for general health information and guidance but not for more specific diagnostics – you’ll need to follow up with a health professional.
• Budget planning – Always ask yourself: will this work for me? Is it realistic and am I comfortable with these suggestions?

For these topics, AI can be a useful starting point, but it’s never a substitute for expert advice or independent checks.

Never rely on AI alone for:

How to check

You might start with a simple checklist:

• Is it accurate?
• Is it current?
• Is it complete?
• Is it unbiased?
• Can I verify it?

Your own judgement

We’ve seen that AI can make mistakes, misunderstand context or give you the wrong information. That's why it's important to check the answers it gives you.

Part of checking these answers is about using your own judgement, to ask yourself whether the information makes sense before acting on it. Not sure how to do this? Start by using the questions in our checklist from the last chapter. 

Ask another human

AI can give quick answers and suggestions to prompts like: ‘I’ve got an hour free this evening – give me some light-hearted options to watch on TV’ or ‘give me some cheap holiday ideas that will work for the whole family’. But sometimes, you’ll need more human guidance. 

Human judgement still matters for those important decisions because they often involve context, ethics and empathy. Here are two examples.

Decisions about your health

Decisions about your money

Want to learn more?