TY - GEN
T1 - Gesture and Voice-Enabled Game Interface for Accessible Human-Computer Interaction
AU - Vasantha Kini, T.
AU - Raj, Tanishk
AU - Meenatchisundaram, S.
AU - Aneesha Acharya, K.
N1 - Publisher Copyright:
© 2025 IEEE.
PY - 2025
Y1 - 2025
N2 - This paper presents a gesture- and voice-controlled, offline-capable, real-time gaming interface inspired by the Chrome Dino game. MediaPipe and OpenCV are integrated into the system for hand tracking, with Kalman filtering used to smooth data on finger angle for accurate gesture recognition. The Vosk engine enables offline voice control and supports commands like”jump” and”exit.” The game, a live camera feed, and real-time overlays displaying finger angles, gesture accuracy, and recognized voice commands are all embedded in a PyQt5-based graphical user interface, which allows switching between control modes. Performance evaluation shows voice recognition success at an average of 66.1% and good gesture accuracy at 85%. The application is packaged as a Windows-compatible standalone executable. Designed with accessibility in mind, this system demonstrates the potential of multimodal interaction for gaming, assistive applications, and user-centric interface design.
AB - This paper presents a gesture- and voice-controlled, offline-capable, real-time gaming interface inspired by the Chrome Dino game. MediaPipe and OpenCV are integrated into the system for hand tracking, with Kalman filtering used to smooth data on finger angle for accurate gesture recognition. The Vosk engine enables offline voice control and supports commands like”jump” and”exit.” The game, a live camera feed, and real-time overlays displaying finger angles, gesture accuracy, and recognized voice commands are all embedded in a PyQt5-based graphical user interface, which allows switching between control modes. Performance evaluation shows voice recognition success at an average of 66.1% and good gesture accuracy at 85%. The application is packaged as a Windows-compatible standalone executable. Designed with accessibility in mind, this system demonstrates the potential of multimodal interaction for gaming, assistive applications, and user-centric interface design.
UR - https://www.scopus.com/pages/publications/105030048833
UR - https://www.scopus.com/pages/publications/105030048833#tab=citedBy
U2 - 10.1109/DISCOVER66922.2025.11258930
DO - 10.1109/DISCOVER66922.2025.11258930
M3 - Conference contribution
AN - SCOPUS:105030048833
T3 - 2025 IEEE International Conference on Distributed Computing, VLSI, Electrical Circuits and Robotics, DISCOVER 2025 - Proceedings
SP - 776
EP - 782
BT - 2025 IEEE International Conference on Distributed Computing, VLSI, Electrical Circuits and Robotics, DISCOVER 2025 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 9th IEEE International Conference on Distributed Computing, VLSI, Electrical Circuits and Robotics, DISCOVER 2025
Y2 - 17 October 2025 through 18 October 2025
ER -