Optimising Artificial Intelligence for Developing Reliable Academic Assessment Design

Artificial intelligence is revolutionising educational practices, particularly in assessment design, offering objectivity, efficiency, and scalability. However, concerns about reliability, fairness, and effectiveness persist among educators. This study explores lecturers’ perceptions of AI-generated academic assessment design and gathers their insights on potential improvements to enhance the reliability and usability of these tools. A qualitative approach was adopted, involving in-depth interviews with twelve (12) lecturers from the Faculty of Technology and Applied Sciences at Open University Malaysia. Participants were selected based on their experience with AI-generated assessment design. Semi-structured interviews provided rich qualitative data, which were audio-recorded, transcribed, and analysed thematically to identify key trends and perspectives. The findings highlight that AI tools can reduce educators’ workloads by automating repetitive tasks, but require significant human input to refine assessments, particularly for evaluating higher-order thinking skills such as critical analysis and problem-solving. Participants emphasised the importance of user training, access to advanced AI systems, and collaborative support for successful implementation. They noted that while AI tools excel at structuring assessments and generating large question banks, they fall short in handling nuanced or context-specific scenarios without human oversight. These findings offer actionable insights for enhancing AI assessment tools, suggesting the integration of user feedback loops, improved algorithms, and institutional support systems. By addressing these areas, AI can contribute to more reliable, fair, and effective assessment methods, ultimately improving educational practices and academic standards at Open University Malaysia and other higher education institutions.]

https://doi.org/10.64233/EDQZ9625