In a groundbreaking experiment, researchers tested the capabilities of artificial intelligence (AI) by subjecting top AI models to the CPA Exam.
The study, "Can LLMs Pass the CPA Exam? Evaluating Large Language Model Performance on the Certified Public Accountant test," provides a comprehensive assessment of AI's potential in the accounting profession.
The research team evaluated the performance of AI models like Google Gemini, ChatGPT-4, Claude, Mixtral and Llama-2b on multiple-choice questions from the Becker CPA Exam preparation suite.
While some AI models showed promise, human accountants still perform better at tackling complex, open-ended questions.
ChatGPT-4 excelled in financial reporting and regulation, while Claude Opus performed well in auditing. Llama-2b underperformed, indicating the need for further development. Learn more.