Government reliance on AI in research has hit a serious snag. The recent Deloitte report on welfare delivery, commissioned by the federal government, was found to contain dozens of AI-generated errors, including fake citations, non-existent legislation, and misattributed sources, raising concerns about oversight, accuracy, and accountability.

Lessons from the Deloitte Report Debacle

The Deloitte report debacle underscores the dangers of relying on AI without robust human verification. AI tools can generate convincing but false citations, and without rigorous cross-checking, errors can propagate into official policy advice.

To prevent similar failures, government contracts should mandate independent quality assurance, including peer review of references and factual claims. Clear accountability measures are also essential, requiring contractors to take full responsibility for any errors and issue timely, public corrections. Departments should invest in internal expertise to scrutinize reports before publication, particularly when AI tools are employed. Finally, agencies could establish AI usage guidelines and risk assessments for commissioned work, ensuring outputs are reliable and verifiable before they inform policy.

These measures would protect taxpayers, maintain trust in government research, and prevent AI hallucinations from undermining critical decision-making.