Multi-task Autonomous Driving Based on Improved Convolutional Neural Network and ST Loss in MTS and MOD Modes
Article Sidebar
Main Article Content
Abstract
Multi-task autonomous driving is a research hotspot in autonomous driving. However, existing research has only achieved single-task or dual-task autonomous driving. Therefore, we propose two novel multi-task approaches: a multi-task shared model mode (MTS) and a multi-object dual-model mode (MOD). In addition, existing neural network architectures are underperforming in multi-task autonomous driving, so we propose a novel neural network architecture - MT-ResNet26. Moreover, to alleviate the problem of noise and class imbalance from data, we propose a new loss function - Stable Loss (ST Loss). Finally, our smart car can achieve multi-task road tracking, left-right turn sign recognition, automatic obstacle avoidance, stop, real-time acceleration and deceleration. In addition, we compare the existing multi-task autonomous driving model YS-VGG17_MSE, which shows our MT-ResNet26_ST is superior in loss value and actual performance. Meanwhile, we use our proposed approaches to train two classical neural networks—ResNet18_MSE* and DenseNet121_MSE*, so that they also achieve multi-task autonomous driving with our proposed approaches, showing the applicability of MTS and MOD. Furthermore, we compare MT-ResNet26_MSE with MT-ResNet26_ST, and the results show that the model using our novel ST Loss outperforms the model using the original loss function MSE. To sum up, it is shown that the performance of multi-task autonomous driving can be achieved and improved using our proposed neural network architecture and loss function. Furthermore, we propose optimized multi-task modes. OMTS and OMOD optimize and accelerate the models using semi-precision techniques based on the TensorRT. The results show that the optimized multi-task autonomous driving accuracy has been further improved.
Keywords: deep learning; loss function; MT-ResNet26; MTS and MOD mode; multi-task autonomous driving
*Corresponding author: Tel.: (+66) 0863759307
E-mail: jianqu@pim.ac.th
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Copyright Transfer Statement
The copyright of this article is transferred to Current Applied Science and Technology journal with effect if and when the article is accepted for publication. The copyright transfer covers the exclusive right to reproduce and distribute the article, including reprints, translations, photographic reproductions, electronic form (offline, online) or any other reproductions of similar nature.
The author warrants that this contribution is original and that he/she has full power to make this grant. The author signs for and accepts responsibility for releasing this material on behalf of any and all co-authors.
Here is the link for download: Copyright transfer form.pdf
References
Hossain, N., Kabir, M.T., Rahman, T.R., Hossen, M.S. and Salauddin, F., 2010. A real-time surveillance mini-rover based on OpenCV-Python-JAVA using Raspberry Pi 2. Proceedings of the 2005 IEEE International Conference on Control System, Computing and Engineering (ICCSCE), Penang, Malaysia, November 27-29 , 2015, pp. 476-481.
Yılmaz, E. and Tarıyan Özyer, S., 2019. Remote and autonomous controlled robotic car based on Arduino with real time obstacle detection and avoidance. Universal Journal of Engineering Science, 7(1), DOI: 10.13189/ujes.2019.070101.
Iqbal, A., Ahmed, S.S., Tauqeer, M.D., Sultan, A. and Abbas, S.Y., 2017. Design of multifunctional autonomous car using ultrasonic and infrared sensors. Proceedings of the 2017 International Symposium on Wireless Systems and Networks (ISWSN), Lahore, Pakistan, November 19-22, 2017, pp. 1-5.
Banerjee, A., Bolar, V., Chaurasia, A., Maurya, S. and Gite, Y., 2020. Autonomous driving vehicle. International Research Journal of Engineering and Technology (IRJET), 7(4), 6048-6050.
Yuenyong, S. and Qu, J., 2017. Generating synthetic training images for deep reinforcement learning of a mobile robot. Journal of Intelligent Informatics and Smart Technology, 2(3), 16-20.
Rausch, V., Hansen, A., Solowjow, E., Liu, C., Kreuzer, E. and Hedrick, J.K., 2017. Learning a deep neural net policy for end-to-end control of autonomous vehicles. Proceedings of the 2017 American Control Conference (ACC), WA, USA, May 24-26, 2017, pp. 4914-4919.
Bechtel, M.G., McEllhiney, E., Kim, M. and Yun, H., 2018. Deeppicar: A low-cost deep neural network-based autonomous car. Proceedings of the 2018 IEEE 24th International Conference on Embedded and Real-time Computing Systems and Applications (RTCSA), Hakodate, Japan, August 28-31, 2018, pp. 11-21.
Meyer, G.P. and Thakurdesai, N., 2020. Learning an uncertainty-aware object detector for autonomous driving. Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA, October 24, 2020 - January, 24, 2021, pp. 10521-10527.
Ren, X., Yang, T., Li, L.E., Alahi, A. and Chen, Q., 2021. Safety-aware motion prediction with unseen vehicles for autonomous driving. Proceedings of the IEEE/CVF International Conference on Computer Vision, Virtual, October 10-17, 2021, pp. 15731-15740.
Do, T.D., Duong, M.T., Dang, Q.V. and Le, M.H., 2018. Real-time self-driving car navigation using deep neural network. Proceedings of the 2018 4th International Conference on Green Technology and Sustainable Development (GTSD), Ho Chi Minh City, Vietnam, November 23-24, 2018, pp. 7-12.
Al-Nima, R.R.O., Han, T. and Chen, T., 2019. Road tracking using deep reinforcement learning for self-driving car applications. Proceedings of the International Conference on Computer Recognition Systems, Polanica-Zdroj, Poland, May 20-22, 2019, pp. 106-116.
Kulkarni, R., Dhavalikar, S. and Bangar, S., 2018. Traffic light detection and recognition for self driving cars using deep learning. Proceedings of the 2018 Fourth International Conference on Computing Communication Control and Automation (ICCUBEA), Pune, India, August 16-18, 2018, pp. 1-4.
Yang, Z., Li, J. and Li, H., 2018. Real-time pedestrian and vehicle detection for autonomous driving. Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China, June 26-30, 2018, pp. 179-184.
Fang, C.Y., Chen, S.W. and Fuh, C.S., 2003. Road-sign detection and tracking. IEEE Transactions on Vehicular Technology, 52(5), 1329-1341, DOI: 10.1109/TVT.2003.810999.
Mitsch, S., Ghorbal, K. and Platzer, A., 2013. On provably safe obstacle avoidance for autonomous robotic ground vehicles. Proceedings of the Robotics: Science and Systems IX, Berlin, Germany, June 24-28, 2018, pp. 14-21.
Kang, D.H., Bong, J.H., Park, J. and Park, S., 2017. Reinforcement learning strategy for automatic control of real-time obstacle avoidance based on vehicle dynamics. Journal of Korea Robotics Society,12(3), 297-305, DOI: 10.7746/jkros.2017.12.3.297.
Tao, L., Hong, T., Guo, Y., Chen, H. and Zhang, J., 2020. Drone identification based on CenterNet-TensorRT. Proceedings of the 2020 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Paris, France, October 27-29, 2020, pp. 1-5.
Ding, S. and Qu, J., 2022. Smart car with road tracking and obstacle avoidance based on Resnet18-CBAM. Proceedings of the 2022 7th International Conference on Business and Industrial Research (ICBIR), Bangkok, Thailand, May 19-20, 2022, pp. 582-585.
Willmott, C.J. and Matsuura, K., 2005. Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate Research, 30(1), 79-82, DOI: 10.3354/cr030079.
Sutanto, A.R. and Kang, D.K., 2021. A Novel Diminish Smooth L1 Loss Model with Generative Adversarial Network. Proceedings of the 12th International Conference on Intelligent Human Computer Interaction, Daegu, South Korea, November 24-26, 2020, pp. 361-368.
Mehta, S., Paunwala, C. and Vaidya, B., 2019. CNN based traffic sign classification using Adam optimizer. Proceedings of the 2019 International Conference on Intelligent Computing and Control Systems (ICCS), Madurai, India, May 15-17, 2019, pp. 1293-1298.
Suo, Y., Chen, S. and Zheng, M., 2020. Developing an Autonomous Driving Model Based on Raspberry Pi. Proceedings of the 53rd AnnualMidwest Instruction and Computing Symposium, Milwaukee Wisconsin, USA, April 3-4, 2020, pp. 56-66.
He, K., Zhang, X., Ren, S. and Sun, J., 2016. Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA, June 27-30, 2016, pp. 770-778.
Huang, G., Liu, Z., Van Der Maaten, L. and Weinberger, K.Q., 2017. Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, July 21-26, 2017, pp. 4700-4708.
Kocić, J., Jovičić, N. and Drndarević, V., 2019. An end-to-end deep neural network for autonomous driving designed for embedded automotive platforms. Sensors, 19(9), 2064-2090, DOI: 10.3390/s19092064.
Li, Y. and Qu, J., 2022. Intelligent road tracking and real-time acceleration-deceleration for autonomous driving using modified convolutional neural networks. Current Applied Science and Technology, 22(6), 199-225.