Keywords
Kenyan Sign Language
deaf communities
hand gestures
communication gap
How to Cite
Abstract
The barrier between the hearing and the deaf communities in Kenya is a major challenge leading to a major gap in the communication sector where the deaf community is left out leading to inequality. The study used primary and secondary data sources to obtain information about this problem, which included online books, articles, conference materials, research reports, and journals on sign language and hand gesture recognition systems. To tackle the problem, CNN was used. Naturally captured hand gesture images were converted into grayscale and used to train a classification model that is able to identify the English alphabets from A-Z. Then identified letters are used to construct sentences. This will be the first step into breaking the communication barrier and the inequality. A sign language recognition model will assist in bridging the exchange of information between the deaf and hearing people in Kenya. The model was trained and tested on various matrices where we achieved an accuracy score of a 99% value when run on epoch of 10, the log loss metric returning a value of 0 meaning that it predicts the actual hand gesture images. The AUC and ROC curves achieved a 0.99 value which is excellent.
References
J. G. Mweri, “Diversity in education: Kenyan sign language as a medium of instruction in schools for the deaf in Kenya,” Multilingual Education, vol. 4, no. 1, p. 14, Aug. 2014, doi: 10.1186/s13616-014-0014-1
R. G. Brill and Conference of Educational Administrators Serving the Deaf, The Conference of Educational Administrators Serving the Deaf: a history. Washington, D.C.: Gallaudet College Press, 1986.
D. M. Perlmutter, “The Language of the Deaf.” Accessed: Nov. 28, 2021. [Online]. Available: https://www.nybooks.com/articles/1991/03/28/the-language-of-the-deaf/
DR. M. G. JEFWA, Mweri, J.G. In Okombo et. al “Introduction to Theory and Skills of Teaching Kenyan sign Language: A handbook for Teachers. Nairobi:,” 2006.
X. Teng. A hand gesture recognition system based on local linear em- bedding, April 2005. Journal of Visual Languages and Computing.
Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, May 2015, doi: 10.1038/nature14539.
J. Ngugi, S. Kimotho, and S. Muturi, “SOCIAL MEDIA USE BY THE DEAF IN BUSINESS AT NAIROBI, KENYA,” vol. 4, no. 3, p. 13, 2018.
E. Keogh, L. Wei, X. Xi, S.-H. Lee, and M. Vlachos, “LB_Keogh Supports Exact Indexing of Shapes under Rotation Invariance with Arbitrary Representations and Distance Measures.,” Jan. 2006, pp. 882–893.
Z. Ren, J. Yuan, and Z. Zhang, “Robust hand gesture recognition based on finger-earth mover’s distance with a commodity depth camera,” Nov. 2011, pp. 1093–1096. doi: 10.1145/2072298.2071946.
Z. Ren, J. Yuan, C. Li, and W. Liu, “Minimum near-convex decomposition for robust shape representation,” in 2011 International Conference on Computer Vision, Nov. 2011, pp. 303–310. doi: 10.1109/ICCV.2011.6126256.
C. Doppler and F. J. Studnica, “Ueber das farbige licht der doppelsterne und einiger anderer gestirne des himmels.,” undefined, Accessed: Nov. 28, 2021. [Online]. Available: https://www.semanticscholar.org/paper/Ueber-das-farbige-licht-der-doppelsterne-und-des-Doppler-Studnica/62bfc7c7b751aacd02d3b4405ace10aedb2d7987
S. Gupta, D. Morris, S. Patel, and D. Tan, “SoundWave: using the doppler effect to sense gestures,” May 2012, doi: 10.1145/2207676.2208331.
L. R. Rabiner, “A tutorial on hidden Markov models and selected applications in speech recognition,” Proceedings of the IEEE, vol. 77, no. 2, pp. 257–286, Feb. 1989, doi: 10.1109/5.18626.
L. Rabiner and B. Juang, “An introduction to hidden Markov models,” IEEE ASSP Magazine, vol. 3, no. 1, pp. 4–16, Jan. 1986, doi: 10.1109/MASSP.1986.1165342.
J. R. Chowdhury, “Kinect Sensor for Xbox Gaming,” p. 39.
“About open Frameworks.” https://openframeworks.cc/about/ (accessed Nov. 28, 2021).
G. R. Bradski and A. Kaehler, Learning OpenCV: computer vision with the OpenCV library, 1. ed., [Nachdr.]. Beijing: O’Reilly, 2011.
OpenNI, “OpenNI recognized by PCL as the standard for 3D sensing acquisition.” https://www.prnewswire.com/news-releases/openni-recognized-by-pcl-as-the-standard-for-3d-sensing-acquisition-190002261.html (accessed Nov. 28, 2021).
D. King, “Dlib-ml: A Machine Learning Toolkit,” Journal of Machine Learning Research, vol. 10, pp. 1755–1758, Jul. 2009, doi: 10.1145/1577069.1755843.
D. Zhang and G. Lu, “A comparative study on shape retrieval using Fourier descriptors with Different shape signatures,” J Vis Commun Image Represent, vol. 1, Jan. 2001
P. Trigueiros, F. Ribeiro, and L. Reis, “A Comparative Study of different image features for hand gesture machine learning,” Jan. 2013, vol. 2.
“Terms of Use | Kaggle.” https://www.kaggle.com/terms (accessed Nov. 28, 2021).
A. Dangi, “‘A new approach for user identification in web usage mining preprocessing,’” IOSR Journal of Computer Engineering, vol. 11, pp. 57–61, Jan. 2013, doi: 10.9790/0661-1135761.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.