Emotion recognition of students during e-learning through online conference meeting
Article Sidebar
Main Article Content
Abstract
Due to the outbreak of COVID-19, online learning has become a way of life. The objective of this study was to propose techniques to detect students’ emotions while studying via online video conferencing. This proposed technique, which updates the facial emotion image of the current class-member, enables the system to achieve a highly accurate performance for facial emotion recognition. This proposed technique can be applied to online teaching systems. As a result, instructors can identify the interest levels of each learner using the interest assessment system, which measures and monitors the period of tiresomeness of each learner. The results showed that our techniques achieved a high percentage of accuracy for each emotion, that is, sleepy/bored = 93.3%, confused = 94.3%, neutral = 92.6%, and happy = 97.2%, which was higher than the convolutional neural network-based emotion recognition system. The proposed system was applied to a real class and satisfactory overall results of 88.7% were achieved. This study proved the feasibility of the proposed technique.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
Al-Samarraie, H. (2019). A scoping review of videoconferencing systems in higher education: Learning paradigms, opportunities, and challenges. International Review of Research in Open and Distributed Learning, 20(3), 121-140.
Chick, R. C., Clifton, G. T., Peace, K. M., Propper, B. W., Hale, D. F., Alseidi, A. A., and Vreeland, T. J. (2020). Using technology to maintain the education of residents during the COVID-19 pandemic. Journal of Surgical Education, 77(4), 729-732.
Cucinotta, D., and Vanelli, M. (2020). WHO declares COVID-19 a pandemic. Acta Bio-medica: Atenei Parmensis, 91(1), 157-160.
Dewan, M. A. A., Murshed, M., and Lin, F. (2019). Engagement detection in online learning: a review. Smart Learning Environments, 6(1), 1-20.
Dwivedi, Y. K., Hughes, D. L., Coombs, C., Constantiou, I., Duan, Y., Edwards, J. S., Gupta, B., Lal, B., Misra, S., Prashant, P., Raman, R., Rana, N. P., Sharms, S. K., and Upadhyay, N. (2020). Impact of COVID-19 pandemic on information management research and practice: transforming education, work and life. International Journal of Information Management, 55, 102211.
Gillies, D. (2007). Student perspectives on videoconferencing in teacher education at a distance. Distance Education, 29(1), 107-118.
Gonzales-Zamora, J. A., Alave, J., De Lima-Corvino, D. F., and Fernandez, A. (2020). Videoconferences of infectious diseases: an educational tool that transcends borders. a useful tool also for the current COVID-19 pandemic. Le Infezioni in Medicina, 28(2), 135-138.
Goodfellow, I. J., Erhan, D., Carrier, P. L., Courville, A., Mirza, M., Hamner, B., Cukierski, W., Tang, Y., Thaler, D., Lee, D. H., Zhou, Y., Ramaiah, C., Feng, F., Li, R., Wang, X., Athanasakis, D., Shawe-Taylor, J., Milakov, M., Park, J., . . . Bengio, Y. (2013). Challenges in representation learning: a report on three machine learning contests. In Neural Information Processing. International Conference on Neural Information Processing (Lee M., Hirose A., Hou ZG. and Kil R. M., eds.), pp. 117-124. Berlin: Springer-Verlag Berlin Heidelberg.
Krithika, L. B., and Lakshmi Priya, G. G. (2016). Student emotion recognition system (SERS) for e-learning improvement based on learner concentration metric. Procedia Computer Science, 85, 767-776.
Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2017). Imagenet classification with deep convolutional neural networks. Communications of the Association for Computing Machinery, 60(6), 84-90.
Marinoni, G., Van’t Land, H., and Jensen, T. (2020). IAU global survey report. In The impact of Covid-19 on higher education around the world, pp. 1-50. Paris: International Association of Universities.
Marzban, C. (2004). The ROC Curve and the Area Under it as Performance Measures. Weather and Forecasting, 19(6), 1106-1114.
Putra, W., and Arifin, F. (2019). Real-time emotion recognition system to monitor student’s mood in a classroom. Journal of Physics: Conference Series, 1413(1), 012021.
Rehn, N., Maor, D., and McConney, A. (2017). Navigating the challenges of delivering secondary school courses by videoconference: Navigating challenges of videoconference. British Journal of Educational Technology, 48(3), 802-813.
Sahla, K., and Kumar, T. S. (2016). Classroom teaching assessment based on student emotions. In Proceedings of the International Symposium on Intelligent Systems Technologies and Applications, pp. 475-486. Jaipur, India.
Soltani, M., Zarzour, H., and Babahenini, M. C. (2018). Facial emotion detection in massive open online courses. In Proceedings of the World Conference on Information Systems and Technologies, pp. 277-286. Napies, Italy.
Tape, T. G. (2013). The area under an ROC curve. Interpreting Diagnostic Tests. [URL: https://www.mathworks.com/hardware-support/home.html?s_tid=srchtitle_MATLAB%2520%2526%2520Simulink_1] accessed on March 18, 2021.
The MathWorks, Inc. (2021). Convolutional neural network. MATLAB & Simulink. [URL: https://www.mathworks.com/discovery/convolutional-neural-networkmatlab.html] accessed on March 18, 2021.
Viola, P., and Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 511-518. Hawaii, USA.
Wang, W., Xu, K., Niu, H., and Miao, X. (2020). Emotion recognition of students based on facial expressions in online education based on the perspective of computer simulation. Complexity, 2020, 4065207.
Zhao, X., and Wei, C. (2017). A real-time face recognition system based on the improved LBPH algorithm. In Proceedings of 2017 IEEE 2nd International Conference on Signal and Image Processing (ICSIP), pp. 72-76. Singapore.