On August 19th, Dr. Sevgi Z. Gurbuz gave a CREST talk entitled “RF Sensing for ASL-based Human-Computer Interactions.” Registration information is given in the talk flyer.
Abstract:
The past decade has seen great advancements in speech recognition for control of interactive devices, personal assistants, and computer interfaces. However, this technology does not address the needs of Deaf and hard-of hearing (HoH) individuals whose primary mode of communication is sign language. Although there has been a large amount of significant work in video-based sign language recognition, video has raised privacy concerns when used in the context of human ambient intelligence. RF sensors have recently been proposed as a new modality for ASL recognition as it has a number of advantageous properties for enabling human-computer interactions (HCI): namely, it is non-contact, effective in the dark, and does not acquire imagery of the user or the environment so that privacy is more protected. RF sensors, also known as radar, short for radio detection and ranging, acquire independent measurements of distance, velocity and angle, which can be used to extract features for ASL recognition that effectively capture motion-related signing dynamics. This talk will present the results of recent research in RF sensing based ASL recognition, our vision for future research in sign language processing technology, and some of our on-going initiatives related to community partnerships and STEM education that leverage RF sensing based ASL recognition.