Certainty-based marking in multiple-choice assessments in physiology: a web-based implementation using an AI assistant

Certainty-based marking (CBM) requires students to indicate their certainty levels alongside their answers. CBM has been shown to enhance self-assessment and metacognitive awareness. This study aimed to explore the implementation of CBM in multiple-choice assessments in physiology. The CBM assessment tool was developed with an artificial intelligence (AI) assistant, Claude 3.5, with prompts focused on functional rather than technical requirements. The assessment consisted of 15 multiple-choice questions (MCQs), which were administered as a pretest and posttest during a small group teaching session to first-year medical students. Following the assessment, students completed a survey to evaluate their perceptions regarding the format, knowledge-gap identification, and overall acceptability. Answers from 195 students were analyzed, and significant improvements were observed in performance measures and certainty indices from the pretest to the posttest. Most students (80.9 NEW & NOTEWORTHY To introduce certainty-based marking (CBM) to novice students, a custom web-based multiple-choice question (MCQ) test was developed with assistance from an artificial intelligence (AI) tool. This enhanced accessibility and allowed for data collection to evaluate and analyze student performance. The integration of AI in creating this assessment tool highlights the potential of technology to improve educational practices, especially in designing various assessment strategies.