Fingerspelling is a gesture based animation app for teaching Indian Sign Language (ISL) for the hearing and speech impaired, developed at IIT Bombay. Fingerspelling consists of ‘Learn’ module: which demonstrates the ISL gestures; Practice module: which provides opportunity to the users to practice; and Test module: which checks the knowledge assimilation of the gestures. It is an open source project.
This was presented in the 2018 IEEE 18th International Conference on Advanced Learning Technologies (ICALT). Click here to find the published paper.
Our preliminary user tests reveal that the hearing impaired students find this app extremely useful and friendly to use. They were pleasantly surprised to interact with an app which can speak ‘their’ language.
This application is not just a animated gesture based library, but also has computer vision in the TEST module to check if the learner/user performs the gestures correctly. Click here to go to the GitHub Link.
Also, I demonstrated this project during a television interview. Check out the video below. (English subtitles available)