Practical AI Hallucination Test Cases for QA Engineers
The following examples can be used when testing an AI chatbot that answers questions about a company’s approved knowledge base.
| Test Case | Test Input | Expected Behavior |
|---|---|---|
| TC01: Correct information | Ask about a documented company policy | The answer matches the approved policy |
| TC02: Missing information | Ask about an undocumented policy | The chatbot states that the information is unavailable |
| TC03: False assumption | Ask a question containing an incorrect technical statement | The chatbot corrects the false assumption |
| TC04: Incorrect numbers | Ask for a documented fee or refund amount | The response gives the correct value |
| TC05: Outdated information | Ask about a recently updated policy | The chatbot uses the current approved information |
| TC06: Unsupported claim | Ask for a fact that is not supported by the reference documents | The chatbot avoids presenting an unsupported claim as fact |
| TC07: Ambiguous question | Ask a question without enough context | The chatbot asks for clarification or explains the ambiguity |
| TC08: Repeated question | Submit the same important question several times | Responses remain factually acceptable |
| TC09: Source verification | Ask for a policy explanation with supporting evidence | The response provides relevant, verifiable source information when available |
| TC10: Conflicting documents | Ask about information that differs between reference sources | The chatbot identifies the conflict or communicates uncertainty |
These test cases are starting points. QA engineers should customize them based on the application’s purpose, risk level, available reference data, and expected behavior.