Assessment of fraudulent conduct during online exams using an artificial intelligence-based automatic student facial expression recognition system
Article Sidebar
Main Article Content
Abstract
The primary issues arising from the disruption of traditional face-to-face examinations in the exam room and the shift to online exams are the unique facial expressions and behaviors of the students, which are difficult to understand with humans. The purposes of this research were to 1) develop a model of educational innovation for the assessment of fraudulent conduct during online exams, using Artificial Intelligence (AI) based on an Automatic Student Facial Expression Recognition (ASFER) system; 2) evaluate the effectiveness of the model through metrics such as accuracy, precision, recall, and F-measure; and 3) evaluate the effectiveness of the model in assessing fraudulent conduct during online exams, using AI based on an ASFER system during its actual use. The results revealed that 1) the model that was developed addressed the problem based on a deep learning method in artificial intelligence, consisting of six steps: data collection from online exam videos, data preparation by extracting frames, development of the Automatic Student Facial Expression Recognition Model, also known as STOU-ASFER using two algorithms: a convolutional neural network (CNN) and a multilayer perceptron (MLP) for classifying the results into those exhibiting a regular face and those exhibiting a face showing signs of fraudulent conduct, evaluation of the model using the four main metrics of accuracy, precision, recall, and F-measure, parameter optimization, and deployment for real-time alerts and summary reporting. 2) The evaluation of the model showed an accuracy value of 86.2%, a precision value of 77.34%, a recall value of 95.7%, and an F-measure of 85.6%, and 3) the evaluation of the model performed well in predicting fraudulent behavior in the simulated examination environment. However, the model needs to be improved for face detection when faces are randomly positioned and when small image sizes are encountered.
Article Details
References
มหาวิทยาลัยสุโขทัยธรรมาธิราช. (2566, 5 พฤษภาคม). ระเบียบและแนวปฏิบัติในการสอบออนไลน์. https://www.stou.ac.th/offices/ore/rere/goto/newsfeed353
Abdullah, M., Ahmad, M., & Han, D. (2020). Facial expression recognition in videos: An CNN-LSTM based model for video classification. 2020 International Conference on Electronics, Information, and Communication (ICEIC), 1–3.
Aggarwal, C. (2018). Neural networks and deep learning. Springer.
Bargal, A., Barsoum, E., Ferrer, C., & Zhang, C. (2016, October). Emotion recognition in the wild from videos using images. In Proceedings of the 18th ACM International Conference on Multimodal Interaction (pp. 433–436).
Ekman, P. (1992). Facial expressions of emotion: An old controversy and new findings. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences, 335(1273), 63–69.
Ekundayo, O. S., & Viriri, S. (2021). Facial expression recognition: A review of trends and techniques. IEEE Access, 9, 136944–136973.
Frank, M. G. (2001). Facial expressions. In International encyclopedia of the social & behavioral sciences (pp. 5230–5234).
Gavade, P. A., Bhat, V. S., & Gavade, A. B. (2023). Learning face expression features from video using spatio-temporal feature extractor and CNN-LSTM. 2023 Seventh International Conference on Image Information Processing (ICIIP), 46–50.
Google AI for Developers. (2022, November 11). Face landmark detection guide. https://ai.google.dev/edge/mediapipe/solutions/vision/face_landmarker
Guodong, G., & Na, Z. (2019). A survey on deep learning-based face recognition. Computer Vision and Image Understanding, 189, 1–37.
Haghpanah, M., Saeedizade, E., Masouleh, M. T., & Kalhor, A. (2022). Real-time facial expression recognition using facial landmarks and neural networks. 2022 International Conference on Machine Vision and Image Processing (MVIP), 1–7.
Indolia, S., Kumar, A., Goswami, M., & Pooja, A. (2018). Conceptual understanding of convolutional neural network- A deep learning approach. Procedia Computer Science, 132, 679–688.
Iqbal, T., Ali, T., Shaf, A., & Ali, M. S. (2023). Enhancing online exam security: Deep learning algorithms for cheating detection. 2023 International Conference on Frontiers of Information Technology (FIT), 126–131.
Lapan, M. (2020). Deep reinforcement learning hands-on (2nd ed.). Packt Publishing.
Mohamad Nezami, O., Dras, M., Hamey, L., Richards, D., Wan, S., & Paris, C. (2020). Automatic recognition of student engagement using deep learning and facial expression. In Machine learning and knowledge discovery in databases. ECML PKDD 2019. Lecture notes in computer science (Vol. 11908, pp. 273–289). Springer.
Noorbehbahani, F., Mohammadi, A., & Aminazadeh, M. (2022). A systematic review of research on cheating in online exams from 2010 to 2021. Education and Information Technologies, 27(6), 8413–8460.
Panlima, A., & Sukvichai, K. (2023). Investigation on MLP, CNNs and Vision Transformer models performance for extracting a human emotions via facial expressions. Proceedings of the IEEE International Conference on Instrumentation, Control, Artificial Intelligence, and Robotics (ICA-SYMP), 127–130.
Ramzan, M., Abid, A., Bilal, M., Aamir, K. M., Memon, S. A., & Chung, T.-S. (2024). Effectiveness of pre-trained CNN networks for detecting abnormal activities in online exams. IEEE Access, 12, 21503–21519.
Soltane, M., & Laouar, M. R. (2021). A smart system to detect cheating in the online exam. 2021 International Conference on Information Systems and Advanced Technologies (ICISAT), 1–5.
Zhang, C., Pan, X., Huapeng, L., Gardiner, A., Sargent, I., Hare, J., & Atkinson, P. (2018). A hybrid MLP-CNN classifier for very fine resolution remotely sensed image classification. ISPRS Journal of Photogrammetry and Remote Sensing, 140, 133–144.
