AI hallucinations in training content: will it invent a step?

A language model writes by predicting likely words, one after another. NIST, in its 2024 profile for generative AI, calls the result confabulation: content stated with confidence that is wrong or false. NIST notes that this follows from how the models are built, and that it matters most in long answers and in fields that need deep local knowledge. Your procedures are exactly that kind of field. When the source is silent, a fluent sentence that sounds like a rule can fill the gap. It reads well, which is why it slips past a quick review.

Invented content rarely looks strange. It looks like a sensible rule from some other company. It tends to sit where your source is thin, so start your review in these places.

• Numbers: temperatures, times, amounts and limits that the source never stated.
• Names: a form, a tool or a role that your company does not use.
• Order: two steps swapped, or a check moved after the step it should guard.
• Exceptions: a tidy rule for a case your source leaves to the manager.
• General advice: safety or service tips that sound right but are not your rule.
• Quiz answers: a correct answer that depends on a fact the lesson never taught.
• Tone: a firm "always" or "never" where your source says "usually".

Most invented steps start as gaps in the source. A short procedure with no numbers, no exceptions and no stop points invites a tool to fill them in. Before you ask for a draft, spend a few minutes making the source complete. The same work makes the procedure clearer for people, too.

• Write down every number the work depends on, with its unit.
• Name forms, tools and screens exactly as your staff say them.
• List the exceptions you know, and who decides each one.
• Add the points where a person should stop and ask for help.
• Say plainly what the procedure does not cover, so a draft has no reason to guess.

The sources we cite do not compare products, so judge the setup rather than the label. Ask the same questions of any tool, whether it is a general assistant or a training product. Limiting a tool to your approved procedures narrows what it is asked to work from. It does not stop the model behind it from writing by prediction. Whatever the tool, the review step is what protects your staff, so plan time for it.

• Which material does it draft from, and can you limit that to approved sources?
• Can a reviewer see which source each part of the draft came from?
• Who checks the draft before staff see it, and is that check recorded?

Three habits matter more than the model. First, the draft is built only from material you chose, not from the open web or a whole shared drive. Second, someone who knows the work compares the draft with the source before anyone learns from it. Third, every fix goes into the source first, so the next draft does not repeat the error. A fix made only in the lesson disappears the next time the lesson is redrafted. The EPA's guidance on writing SOPs makes a similar point for any procedure. It says drafts should be reviewed by people with experience of the process, and it helps when someone other than the writer tests them.

In Elvina, you add your sources to Company Knowledge and a named person approves each one before use, and both steps are available now. Creation Studio drafts lessons, job aids and quizzes only from those approved sources, and this is available now. Every draft then waits for a person to approve it before publication, and that review step is available now too.

When you change a source, our team starts the redraft of the lessons built on it today, and a person approves the new version before publication. Today our team runs these drafts with you on the platform, and Elvina picks up each step as it is confirmed. Answers to staff in Chat, limited to approved knowledge, are planned, not available today. None of this means errors cannot happen. A model can still misread a source or merge two steps that should stay apart, so the review is real work.

Use this checklist on every draft, with the source open next to it. It works for any drafting tool, not only ours. Spend the most time on the first drafts for a role, while you learn where the tool tends to drift.

The approver is the person who owns the procedure, not the person who asked for the course. Sign-off means they read the draft against the source and accept both. If the approver cannot say where a sentence came from, the sentence waits. If the right approver has no time to read drafts against sources, pick a different approver rather than a faster review. Keep a short record of each approval: the source version, the date and the name. When a mistake shows up later, that record tells you whether the source was wrong or the draft drifted.

A fictional laundromat with a wash-and-fold service writes down its intake procedure for new attendants. The source covers weighing, tagging and checking pockets. It says nothing about delicates.

• The first lesson draft adds a step: wash delicates on cold in a mesh bag.
• The owner reads it next to the source and cannot find that line, because her real rule is different.
• Her rule is to bag delicates separately and call the customer before washing them.
• She adds that rule to the source, approves the new version and has the lesson redrafted.
• The new draft matches the source, and the quiz question on delicates now tests her rule.

Track invented content the way you track any defect: count it, trace it and watch the trend across drafts. These numbers come from your own reviews, so keep the method the same for each role.

• Unsourced statements found per draft, by type: number, name, order, exception or advice.
• Review rounds before each lesson is approved.
• Days from a source change to an approved, redrafted lesson.
• Mistakes at work that trace back to a lesson rather than to the source.
• Questions new hires ask that the approved lesson should have answered.

Will an AI training tool invent a procedure? It can. Generative models sometimes state wrong or unsupported content with confidence, especially where your source is thin. Limit drafts to approved sources, have the owner of the process compare each draft with its source, and fix gaps in the source before redrafting. Review lowers the risk; it does not remove it.

• Can we switch off the AI's general knowledge? Not fully, because the model's general language skill is what lets it write at all. What you can control is the material it drafts from and what reaches staff. Restrict sources to approved documents and keep a person's approval between every draft and every learner.
• What if our procedure itself is wrong or out of date? Then a faithful draft will teach the wrong thing well. Fix the source first, approve the new version, then redraft the lessons that rely on it. Having a newer employee try the procedure once is a cheap way to find gaps before a whole team learns them.

Public sources, checked on the date shown.

• NIST defines confabulation as generative AI confidently presenting erroneous or false content, and says it follows from statistical next-word prediction. Source: https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf (checked 2026-10-02).
• EPA guidance says SOPs should be reviewed by people experienced with the process, ideally tested by someone other than the writer. Source: https://www.epa.gov/sites/default/files/2015-06/documents/g6-final.pdf (checked 2026-10-02).