Over the past year, we had much more time indoors to brush up on hobbies and try out new skills. I had plenty of friends who took up painting or sewing and many took the time to try learning a new language. Thanks to the age of the internet, there are tons of digital resources to help use learn spoken languages. But, what about American Sign Language (ASL)? Sure, there are YouTube videos and plenty of infographics, but, aside from one or two platforms, there isn’t a bevy of games, tools, and sites like you might find if you wanted to learn Spanish or Arabic. However, we just came across an impressive site that uses machine learning to teach the basics of ASL. Fingerspelling is an interactive tool developed by the American Society for Deaf Children and creative studio Hello Monday that uses hand and finger tracking from MediaPipe Hands to teach the ASL alphabet. While the tech is advanced, Fingerspelling is incredibly simple and built to aid deaf children in learning the ABCs. But, anybody can use it!
Simply head to Fingerspelling, turn on your webcam, and select a level between 1-4. A word will flash on your screen and a computer-generated hand will shape each letter. Your job is to match it while the tool maps your hand movements to judge accuracy. It creates a really engaging game where you’re attempting to spell words as quickly as possible, not unlike the classic typing games of grade school computer class. The AI is fairly accurate (some of the more complex hand movements can be tough to map) and the tool definitely makes learning ASL fun and intuitive. Fingerspelling is a simple application of machine learning and we’re excited to see how the technology can be iterated on in the future. Maybe you’ll be able to construct complex sentences in ASL or have fully signed conversations with the AI. Whatever the case, Fingerspelling is a great, free-to-use tool that showcases the educational power of technology.
