A Real-Time Human-Drone Interaction System for Cornfield Perimeter Monitoring Using Hand Gesture Control
DOI:
https://doi.org/10.37802/joti.v8i1.1314Keywords:
Drone Control, Gesture Recognition, MediaPipe Hands, Perimeter Monitoring, Smart Farming, Support Vector Machine (SVM)Abstract
Perimeter monitoring in agricultural fields is essential for maintaining security and ensuring continuous observation of field conditions. This study develops a real-time human–drone interaction system using hand-gesture recognition based on MediaPipe Hands and a Support Vector Machine (SVM) classifier. A custom dataset of 24,000 images across 12 gesture classes was collected and converted into 42 hand landmarks (x, y, z), normalized relative to the wrist point. The SVM model with an RBF kernel was trained using an 80:20 split and achieved a testing accuracy of 99.18%. The system operates at 109 FPS with an average latency of 9.16 ms, enabling rapid and reliable drone responses to gesture commands. Field testing in a cornfield with FPV camera visualization demonstrated that the system consistently recognized gestures in varying outdoor lighting, allowing drones to execute precise perimeter checks and maneuvers. These results highlight the significant potential of integrating gesture recognition with drone control, providing a practical, real-world solution that advances smart farming, increases agricultural efficiency, and supports technological progress toward Sustainable Development Goals. The proposed system thus offers a lightweight, responsive, and impactful tool for modern agricultural perimeter monitoring.
Downloads
References
F. A. Almalki, B. O. Soufiene, S. H. Alsamhi, and H. Sakli, “A low-cost platform for environmental smart farming monitoring system based on iot and uavs,” Sustainability (Switzerland), vol. 13, no. 11, Jun. 2021, doi: 10.3390/su13115908.
A. Hafeez et al., “Implementation of drone technology for farm monitoring & pesticide spraying: A review,” Jun. 01, 2023, China Agricultural University. doi: 10.1016/j.inpa.2022.02.002.
M. Yoo et al., “Motion Estimation and Hand Gesture Recognition-Based Human–UAV Interaction Approach in Real Time,” Sensors, vol. 22, no. 7, Apr. 2022, doi: 10.3390/s22072513.
G. Ipate, C. Tudora, and F. Ilie, “Digital Analysis with the Help of an Integrated UAV System for the Surveillance of Fruit and Wine Areas,” Agriculture (Switzerland), vol. 14, no. 11, Nov. 2024, doi: 10.3390/agriculture14111930.
P. Srinil and P. Thongnim, “Deep Learning Enhanced Hand Gesture Recognition for Efficient Drone use in Agriculture,” 2024. [Online]. Available: www.ijacsa.thesai.org
M. Anggraeni, H. Andhika F. R., H. Rante, and S. Sukaridhoto, “Indonesian Sign Language (SIBI) Learning Media Application Based on Deep Learning Technology for Deaf Children,” Journal of Technology and Informatics (JoTI), vol. 5, no. 1, pp. 41–47, Oct. 2023, doi: 10.37802/joti.v5i1.384.
B. Taylor, M. Allen, P. Henson, X. Gao, H. Malik, and P. Zhu, “Enhancing Drone Navigation and Control: Gesture-Based Piloting, Obstacle Avoidance, and 3D Trajectory Mapping,” Applied Sciences (Switzerland), vol. 15, no. 13, Jul. 2025, doi: 10.3390/app15137340.
N. Chalista Imanuela Natun, M. Angelica Santhia, and Y. R. Kaesmetan, “Identifikasi Pengenalan Wajah Berdasarkan Jenis Kelamin Menggunakan Metode Convolutional Neural Network (CNN),” Journal of Technology and Informatics (JoTI), vol. 6, no. 1, pp. 50–57, Oct. 2024, doi: 10.37802/joti.v6i1.694.
M. Fuad et al., “Towards Controlling Mobile Robot Using Upper Human Body Gesture Based on Convolutional Neural Network,” Journal of Robotics and Control (JRC), vol. 4, no. 6, pp. 856–867, 2023, doi: 10.18196/jrc.v4i6.20399.
S. Abdalla and S. Baidya, “UAV Control with Vision-based Hand Gesture Recognition over Edge-Computing,” May 2025, [Online]. Available: http://arxiv.org/abs/2505.17303
2020 IEEE 16th International Conference on Automation Science and Engineering (CASE). IEEE, 2020.
B. Hu and J. Wang, “Deep Learning Based Hand Gesture Recognition and UAV Flight Controls,” International Journal of Automation and Computing, vol. 17, no. 1, pp. 17–29, Feb. 2020, doi: 10.1007/s11633-019-1194-7.
J. W. Lee and K. H. Yu, “Wearable Drone Controller: Machine Learning-Based Hand Gesture Recognition and Vibrotactile Feedback,” Sensors, vol. 23, no. 5, Mar. 2023, doi: 10.3390/s23052666.
D. Tezza and M. Andujar, “The State-of-the-Art of Human-Drone Interaction: A Survey,” IEEE Access, vol. 7, pp. 167438–167454, 2019, doi: 10.1109/ACCESS.2019.2953900.
A. Azka, A. Santoso, and T. Agustinah, “Controlling a Quadcopter with Static Loads and Dynamic Wind Disturbances using a Fuzzy Controller,” 2024.
M. S. Abdallah, G. H. Samaan, A. R. Wadie, F. Makhmudov, and Y. I. Cho, “Light-Weight Deep Learning Techniques with Advanced Processing for Real-Time Hand Gesture Recognition,” Sensors, vol. 23, no. 1, Jan. 2023, doi: 10.3390/s23010002.
A. Edet, S. Inyang, I. Umoren, and U. E. Etuk, “Machine Learning Approach for Classification of Cyber Threats Actors in Web Region,” Journal of Technology and Informatics (JoTI), vol. 6, no. 1, pp. 70–77, Oct. 2024, doi: 10.37802/joti.v6i1.679.
F. Zhang et al., “MediaPipe Hands: On-device Real-time Hand Tracking,” Jun. 2020, [Online]. Available: http://arxiv.org/abs/2006.10214
“Hand landmarks detection guide | Google AI Edge | Google AI for Developers.” Accessed: Oct. 30, 2025. [Online]. Available: https://ai.google.dev/edge/mediapipe/solutions/vision/hand_landmarker
Lukman Arif Sanjani, R. Bimo Mandala Putra, and U. Laili Yuhana, “Exploring the Application of Machine Learning for Automatic Inbound Email Classification in CRM System at XYZ Company,” Journal of Technology and Informatics (JoTI), vol. 6, no. 1, pp. 1–7, Oct. 2024, doi: 10.37802/joti.v6i1.715.
