An-Chih Cheng (Associate Professor, College of Education) presented “AI Powered Exam Prep (Adaptive Testing System)” at the AI in Teaching Symposium, October 18, 2024.
In his presentation, An-Chih Cheng explores how AI can help faculty create adaptive testing systems for improved content preparation and licensure exam preparation.
Challenges with Traditional Testing:
Cheng points out that traditional publisher-provided quizzes and tests come with several problems.
- Students can easily find answers online, which impacts test authenticity and the ability to assess student knowledge.
- Published quizzes might not align with the specific topics emphasized in a particular course.
- These quizzes often focus on basic skills like memorization, rather than higher-order thinking skills like critical thinking.
- Additionally, commercially available licensure exam study materials can be outdated and may not reflect the adaptive format of modern computer-based exams.
The Benefits of AI-Powered Adaptive Testing:
- Cheng argues that AI offers solutions by enabling faculty to create their own test banks and develop computer-adaptive testing systems.
- AI can assist in generating a large volume of high-quality quiz questions, although Cheng emphasizes the importance of prompt engineering and expert review to ensure quality and alignment with testing theories like Bloom’s Taxonomy.
- This customized test bank allows for quizzes tailored to specific course content and licensure exam requirements.
- Using the AI-generated test bank, an adaptive testing system can be created that adjusts the difficulty level of questions based on student responses. This system offers several advantages:
- Precise Measurement: It provides a more accurate assessment of student ability than traditional fixed-form tests.
- Time Efficiency: Adaptive tests can be shorter than traditional tests, as they efficiently target a student’s skill level.
- Immediate Feedback: Students receive real-time feedback on their performance, allowing for timely adjustments and intervention by instructors.
- Reduced Academic Misconduct: As each student receives a unique set of questions, the opportunity for cheating is minimized.
- Simulation of Licensure Exams: Adaptive systems can simulate the experience of real-world computer-based licensure exams, which can help students build confidence and familiarity with the format.
Implementation of the System:
According to Cheng, building an adaptive system involves these key steps:
- Develop a Comprehensive Question Bank: Utilizing AI like ChatGPT, and expert validation to create a large pool of high-quality questions.
- Develop the Adaptive Algorithm: AI can be used to create a program that selects questions based on student performance and adjusts difficulty levels.
- Integrate with Learning Management Systems: The adaptive system should be integrated into existing platforms like D2L or Blackboard for seamless delivery.
Applications Across Disciplines:
Cheng highlights the versatility of this approach, suggesting its applicability to various disciplines and licensure exams, including LSAT preparation for undergraduates, bar exams for law students, and nursing exams.
Overall, Cheng emphasizes that AI-powered adaptive testing has the potential to transform teaching and learning at DePaul. He encourages faculty and staff to consider implementing such systems to enhance student learning, improve exam preparation, and foster a more engaging and effective educational experience.