RF-ASL: RF Sensing of Sign Language

RF Sensing for ASL-based Human-Computer Interactions
Cultivating Research Equity in Sign-related Technology (CREST) Talk
(Presentation in spoken English with ASL interpretation)
Dr. Sevgi Z. Gurbuz / August 19, 2022
[ Flyer ] [ Slides ]

Although there are over 1 million users of American Sign Language (ASL) in the U.S. and Canada, who rely on technology as an assistive device to navigate communication/language barriers that status quo society often creates, many emerging technologies are designed for hearing individuals via voice-controlled devices. This precludes the Deaf/Hard-of-Hearing (HoH) community from benefitting from advances in technology, which, if designed to be compatible with ASL, could in fact generate tangible improvements to their quality of life.

My research aims at transforming ubiquitous sensing technologies through the integration of a new sensing modality – radio frequency (RF) sensing or radar – into smart environments designed to respond to the needs of ASL users.

RF sensors are uniquely desirable for this application because they are 1) non-contact, 2) can operate in the dark or through-the-wall, 3) protect privacy as they cannot acquire visual images of the user or environment, and 4) bring to bear a new type of information that aides in ASL recognition: namely, the micro-Doppler signature, which is reflective of the time-varying velocity profiles of motion. Thus, RF sensing can capture the kinematics of the rapid progression of dynamic sign sequences that is characteristic of ASL usage.