隨著科技與電動車技術的蓬勃發展，近年來電動車獲得更多關注，建設更為便利的充電設施已然是電動車主們一大需求。因此本論文提出一主僕式架構之自動充電系統，提供自動充電服務，整體架構由自主移動式充電機器人與電源拖車組成，並搭配本研究設計之充電行為流程、充電插座姿態辨識與使用深度強化學習(Deep Reinforcement Learning, RL)於機械手臂運動控制，最終實現可提供充電服務之自動充電系統。
本論文於充電插座自動辨識上使用Yolo、PnP等電腦視覺技術，並搭配類神經網路進行座標補償。在機械手臂控制策略上使用深度強化學習之深度確定策略梯度（Deep Deterministic Policy Gradient, DDPG）與近端策略最佳化（Proximal Policy Optimization, PPO）進行模擬實驗，並最終使用PPO搭配PID補償器進行實作，此設計架構可有效補償PPO輸出之穩態誤差，在機械手臂運動控制方面，滿足系統執行自動充電服務的需求。

With the rapid advancement of technology and electric vehicle (EV) technology, EVs have gained significant attention in recent years. The demand for more convenient charging facilities has become a major requirement for EV owners. Therefore, this thesis proposes an automated charging system based on a master-slave architecture, providing automatic charging services. The overall structure consists of a mobile charging robot and a power trailer, combined with a charging behavior process, charging socket posture recognition, and deep reinforcement learning (RL) for motion control of the robotic arm, ultimately achieving an automated charging system that offers charging services.
In this thesis, automatic recognition of the charging socket is performed using computer vision techniques such as Yolo and PnP, along with coordinate compensation through neural networks. For the robotic arm control strategy, Deep Reinforcement Learning algorithms like Deep Deterministic Policy Gradient (DDPG) and Proximal Policy Optimization (PPO) were used for simulation experiments. PPO, combined with a PID compensator, was finally implemented. This design framework effectively compensates for steady-state errors in the PPO output, fulfilling the system’s requirements for executing automated charging services in robotic arm motion control.

[1] Y. Liu, H. Chen, Y. Li, J. Gao, K. Dave, J. Chen, and R. Tu, “Exhaust and non-exhaust emissions from conventional and electric vehicles: A comparison of monetary impact values,” Journal of Cleaner Production, vol. 331, p. 129965, 2022.
[2] A. Albatayneh, M. N. Assaf, D. Alterman, and M. Jaradat, “Comparison of the overall energy efficiency for internal combustion engine vehicles and electric vehicles,” Rigas Tehniskas Universitates Zinatniskie Raksti, vol. 24, no. 1, pp. 669-680, 2020.
[3] J. A. Sanguesa, V. Torres-Sanz, P. Garrido, F. J. Martinez, and J. M. Marquez-Barja, “A review on electric vehicles: Technologies and challenges,” Smart Cities, vol. 4, no. 1, pp. 372-404, Mar. 2021.
[4] Z. A. Lashari, J. Ko, and J. Jang, “Consumers’ intention to purchase electric vehicles: Influences of user attitude and perception,” Sustainability, vol. 13, no. 12, p. 6778, 2021.
[5] “Global EV Outlook 2023 Policy developments” Available: https://www.iea.org/reports/global-ev-outlook-2023/policy-developments
[6] S. Afshar, P. Macedo, F. Mohamed, and V. Disfani, “Mobile charging stations for electric vehicles—A review,” Renewable and Sustainable Energy Reviews, vol. 152, 2021.
[7] T. Chen, X. P. Zhang, J. Wang, J. Li, C. Wu, M. Hu, and H. Bian, “A review on electric vehicle charging infrastructure development in the UK,” Journal of Modern Power Systems and Clean Energy, vol. 8, no. 2, pp. 193-205, 2020.
[8] “KUKA Charging Assistant Charging assistant for electric vehicles,” Available: https://ifdesign.com/en/winner-ranking/project/kuka-charging-assistant/282642
[9] “Autonomous charging of electric vehicles with robotics: How it works,” Available: https://www.rocsys.com/newsroom/autonomous-charging-of-electric-vehicles-with-robotics-how-it-works/
[10] “Electric Vehicle charging with an autonomous robot powered by force sensing|BotaSystems,”Available: https://youtu.be/SrtU32lWu9s?si=dYUT1YE2IzZ2PQSz
[11] “Hyundai Motor Group Shows Newly Developed Automatic Charging Robot for Electric Vehicles,” Available: https://www.hyundai.com/worldwide/en/newsroom/detail/hyundai-motor-group-shows-newly-developed-automatic-charging-robot-for-electric-vehicles-0000000192
[12] “Meet 'CARL' – the Autonomous EV Charging Robot,” Available: https://youtu.be/OOsFz5c8WaU?si=QWXJUPeREmYgNdB3
[13] “Autev Launches Autonomous Charging Robot for EVs,” Available: https://youtu.be/mzUpWVkdoas?si=3ohM_EIBHMYO2EjM
[14] “EVAR Parky (English version),” Available: https://youtu.be/PUOG1_MGlts?si=utWe3NksCZ5_5DQL
[15] “E-HERO移動式充電機器人” Available: https://www.chander.com.tw/e-hero5.html
[16] “汽車充電樁規格懶人包：充電樁種類&架構一篇看懂！,” Available: https://www.zerovatech.com/zh-hant/technical-knowledge/ev-charging-connector-types/
[17] “電動車充電樁的規格、安裝、充電時間及常見問題，完整解析！,” Available: https://www.fubon.com/insurance/blog/information/charging-station.html
[18] “【電動車充電原理】2大充電樁類型規格要了解，快充體驗超方便！,” Available: https://www.zerovatech.com/zh-hant/technical-knowledge/main-types-of-ev-charger-and-application/
[19] “電動車充電系統-介面,” Available: https://www.bsmi.gov.tw/wSite/
[20] D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[21] E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, “ORB: An efficient alternative to SIFT or SURF,” in Proceedings of the 2011 International Conference on Computer Vision (ICCV), pp. 2564-2571, 2011.
[22] R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 580-587, 2014.
[23] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137-1149, 2017.
[24] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, and A. C. Berg, “SSD: Single shot multibox detector,” in Computer Vision–ECCV 2016: 14th European Conference on Computer Vision, vol. 14, pp. 21-37, 2016.
[25] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779-788, 2016.
[26] H. Perez and J. H. Tah, “Towards automated measurement of as-built components using computer vision,” Sensors, vol. 23, no. 16, p. 7110, 2023.
[27] “Perspective-n-Point (PnP) pose computation,” Available: https://docs.opencv.org/4.x/d5/d1f/calib3d_solvePnP.html
[28] M. Pan, C. Sun, J. Liu, and Y. Wang, “Automatic recognition and location system for electric vehicle charging port in complex environment,” IET Image Processing, vol. 14, no. 10, pp. 2263-2272, Oct. 2020.
[29] P. Zhao, X. Chen, S. Tang, Y. Xu, M. Yu, and P. Xu, “Fast recognition and localization of electric vehicle charging socket based on deep learning and affine correction,” in 2022 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 2140-2145, 2022.
[30] H. H. Huang, C. K. Cheng, Y. H. Chen, and H. Y. Tsai, “The robotic arm velocity planning based on reinforcement learning,” International Journal of Precision Engineering and Manufacturing, vol. 24, no. 9, pp. 1707-1721, 2023.
[31] A. V. Duka, “Neural network based inverse kinematics solution for trajectory tracking of a robotic arm,” Procedia Technology, vol. 12, pp. 20-27, 2014.
[32] T. Zhang, M. Xiao, Y. B. Zou, J. D. Xiao, and S. Y. Chen, “Robotic curved surface tracking with a neural network for angle identification and constant force control based on reinforcement learning,” International Journal of Precision Engineering and Manufacturing, vol. 21, pp. 869-882, 2020.
[33] Y. Matsuo, Y. LeCun, M. Sahani, D. Precup, D. Silver, M. Sugiyama, ... and J. Morimoto, “Deep learning, reinforcement learning, and world models,” Neural Networks, vol. 152, pp. 267-275, 2022.
[34] A. G. Barto, R. S. Sutton, and C. W. Anderson, “Neuronlike adaptive elements that can solve difficult learning control problems,” IEEE Transactions on Systems, Man, and Cybernetics, no. 5, pp. 834-846, 1983.
[35] A. Romero, Y. Song, and D. Scaramuzza, “Actor-critic model predictive control,” in 2024 IEEE International Conference on Robotics and Automation (ICRA), May 2024, pp. 14777-14784. IEEE.
[36] R. Liu, F. Nageotte, P. Zanne, M. de Mathelin, and B. Dresp-Langley, “Deep reinforcement learning for the control of robotic manipulation: A focused mini-review,” Robotics, vol. 10, no. 1, p. 22, 2021.
[37] T. Zhang and H. Mo, “Reinforcement learning for robot research: A comprehensive review and open issues,” International Journal of Advanced Robotic Systems, vol. 18, no. 3, 2021.
[38] M. Matulis and C. Harvey, “A robot arm digital twin utilizing reinforcement learning,” Computers & Graphics, vol. 95, pp. 106-114, 2021.
[39] T. Lindner and A. Milecki, “Reinforcement learning-based algorithm to avoid obstacles by the anthropomorphic robotic arm,” Applied Sciences, vol. 12, no. 13, p. 6629, 2022.
[40] “MODBUS APPLICATION PROTOCOL SPECIFICATION V1.1b3,” Available: https://www.modbus.org/
[41] O’Mahony, N., Campbell, S., Carvalho, A., Harapanahalli, S., Hernandez, G. V., Krpalkova, L., & Walsh, J. (2020). Deep learning vs. traditional computer vision. In Advances in Computer Vision: Proceedings of the 2019 Computer Vision Conference (CVC), Volume 1 (pp. 128-144). Springer International Publishing.
[42] C. Y. Wang, A. Bochkovskiy, and H. Y. M. Liao, “YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7464-7475, 2023.
[43] G. Terzakis and M. Lourakis, “A consistently fast and globally optimal solution to the perspective-n-point problem,” in Computer Vision–ECCV 2020: 16th European Conference on Computer Vision, vol. 16, pp. 478-494, 2020.
[44] S. Garrido-Jurado, R. Muñoz-Salinas, F. J. Madrid-Cuevas, and M. J. Marín-Jiménez, “Automatic generation and detection of highly reliable fiducial markers under occlusion,” Pattern Recognition, vol. 47, no. 6, pp. 2280-2292, 2014.
[45] J. Jiang, X. Luo, Q. Luo, L. Qiao, and M. Li, “An overview of hand-eye calibration,” The International Journal of Advanced Manufacturing Technology, vol. 119, no. 1, pp. 77-97, 2022.
[46] I. Enebuse, M. Foo, B. S. K. K. Ibrahim, H. Ahmed, F. Supmak, and O. S. Eyobu, “A comparative review of hand-eye calibration techniques for vision guided robots,” IEEE Access, vol. 9, pp. 113143-113155, 2021.
[47] T. P. Lillicrap, “Continuous control with deep reinforcement learning,” arXiv preprint arXiv:1509.02971, 2015.
[48] J. Schulman, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017.
[49] J. Ren, Y. Lan, X. Xu, Y. Zhang, Q. Fang, and Y. Zeng, “Deep reinforcement learning using least‐squares truncated temporal‐difference,” CAAI Transactions on Intelligence Technology, vol. 9, no. 2, pp. 425-439, 2024.
[50] R. S. Sutton, D. McAllester, S. Singh, and Y. Mansour, “Policy gradient methods for reinforcement learning with function approximation,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 12, 1999.