Creative Circle
Launch the experience

Award submissionFingerspelling with Machine Learning

Introduction

Fingerspelling.xyz is a browser-based app that uses a webcam and machine learning to analyze your hand shapes so you can learn to sign the ASL alphabet correctly.

How it works

To start, you are given a series of words and the computer shows you a 3D model of how your hand should be positioned for each letter. When you sign the word, the camera tracks your hand movements and gives you feedback so you can make corrections as needed. This helps you quickly develop your fingerspelling skills and move to the next level of the program.

The fingerspelling game is a great way to introduce the basics of ASL in a fun and playful way. Instead of having to read or watch videos about fingerspelling, we offer an online teaching tool that guides you step by step in how to master fingerspelling - hands on! The game leverages advanced hand recognition technology, matched with machine learning, to give you real time feedback via the webcam for each sign and word you spell correctly. The game is designed for desktop, primarily to be used by parents for deaf children, but kids will also find it a fun way to practice and improve their fingerspelling technique.

Fingerspelling is an essential part of ASL, the primary language of the Deaf community. It is often used for proper nouns or to sign a word you don’t know the sign for.

Launch the experience