TY - GEN
T1 - Paper piano-Shadow analysis based touch interaction
AU - Vishal, Boga
AU - Lawrence, K. Deepak
N1 - Publisher Copyright:
© 2017 IEEE.
PY - 2018/3/7
Y1 - 2018/3/7
N2 - Recent years have marked a sharp increase in the number of ways in which people interact with electronic devices. Keyboards and computer mouse devices are not the only way users interact with their computers. This paper aims to investigate and demonstrate a human-computer interaction application in which a user can interact with the layout of a piano on a sheet of paper using computer vision. The image is acquired using a general-purpose camera from a single direction. The work presents details about methods of piano key detection and labelling, hand movement detection, fingertip detection and tracking, and touch interaction based on the shadow of the finger. K-means clustering is used to segment the hand and its shadow. Contour analysis is used in determining the shape and position of the keys. Shadow analysis aids in determining if the finger is touching the paper, even without any depth data from the camera. The detected key press then triggers a sound. The paper covers the details of the development of the virtual piano from concept to realization.
AB - Recent years have marked a sharp increase in the number of ways in which people interact with electronic devices. Keyboards and computer mouse devices are not the only way users interact with their computers. This paper aims to investigate and demonstrate a human-computer interaction application in which a user can interact with the layout of a piano on a sheet of paper using computer vision. The image is acquired using a general-purpose camera from a single direction. The work presents details about methods of piano key detection and labelling, hand movement detection, fingertip detection and tracking, and touch interaction based on the shadow of the finger. K-means clustering is used to segment the hand and its shadow. Contour analysis is used in determining the shape and position of the keys. Shadow analysis aids in determining if the finger is touching the paper, even without any depth data from the camera. The detected key press then triggers a sound. The paper covers the details of the development of the virtual piano from concept to realization.
UR - http://www.scopus.com/inward/record.url?scp=85047308758&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85047308758&partnerID=8YFLogxK
U2 - 10.1109/MAMI.2017.8307890
DO - 10.1109/MAMI.2017.8307890
M3 - Conference contribution
AN - SCOPUS:85047308758
VL - 2018-March
T3 - Proceedings - 2017 2nd International Conference on Man and Machine Interfacing, MAMI 2017
SP - 1
EP - 6
BT - Proceedings - 2017 2nd International Conference on Man and Machine Interfacing, MAMI 2017
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2nd International Conference on Man and Machine Interfacing, MAMI 2017
Y2 - 21 December 2017 through 23 December 2017
ER -
