Select Page

Unleashing the Potential of Multi-Modal Wearable Sensing Systems

The recent explosion in the variety and usage of wearable sensing systems is enabling the continuous monitoring of health and wellness of users. Individuals are taking such information and making changes to their personal routines in order to minimize exposures to pollutants and maintain healthy levels of exercise. Furthermore, medical practitioners are using these systems to monitor proper activity levels for rehabilitation purposes and to monitor threatening conditions such as heart arrhythmias. However, there is still much to be done to facilitate the processing and interpretation of such information in order to maximize their impact.

The objective of this proposal is to develop a computational framework that unleashes the potential of physiological and environmental multi-modal wearable systems. The project aims to develop methodology for the estimation and prediction of physiological responses and environmental factors, with the objective of enabling users to efficiently change their behavior. To accomplish this objective, the framework will build on tools from statistical analysis, topological data analysis, optimization theory and human behavior analysis. This novel framework with not only develop new formal techniques, but it will also serve as a bridge between these cross-disciplinary fields. This project will: (1) develop methodology for the concurrent representation of physiological, kinematic and environmental states for inference purposes; (2) develop techniques for mapping representations between different systems to enable information sharing; and (3) develop techniques to maximize the impact on the behavior of individuals by building on the proposed data representation. The proposed techniques will empower users and medical practitioners to understanding, analyze, and make decisions based on patterns present in the data. The outcomes of this project will empower medical practitioners by providing innovative and effective tools for wearable sensing systems which enable efficient pattern identification, data representation and visualization.

This research is funded by the NSF under the CNS 1552828 CAREER Award (April 2016 – March 2021), and some of the work is done in collaboration with the NSF Advanced Self-Powered Systems of Integrated Sensors and Technologies (ASSIST) Center at NCSU.

Students

  • Current Students: Boxuan Zhong (PhD – NCSU), Laura Gonzalez (PhD – NCSU), Tuner Richmond (PhD – NCSU), Nathan Starliper (PhD – NCSU), Rafael da Silva (PhD – NCSU), Jeffrey Barahona (PhD – NCSU).
  • Former Students: Alireza Dirafzoon, PhD; Namita Lokare, PhD.

Resources

  • Lower Limb Prostheses Environmental Context Dataset [Release: May 2020] – This dataset contains motion and visual sensing (from a lower limb and glasses perspectives) for context awareness for lower limb prostheses. Additional details on this project can be found here.
  • NCSU-ADL Dataset [Released: November 2017] – This dataset contains motion and physiological measurements from individuals performing specific activities of daily life at their personal pace.

In the News

Publications

  1. B. Zhong, H. Huang and E. Lobaton, “Reliable Vision-Based Grasping Target Recognition for Upper Limb Prostheses,” in IEEE Transactions on Cybernetics, 2020.
  2. B. Zhong, R. da Silva, M. Li, H. Huang and E. Lobaton, “Environmental Context Prediction for Lower Limb Prostheses with Uncertainty Quantification,” IEEE Transactions on Automation Science and Engineering, 2020.
  3. R. da Silva, E. Stone and E. Lobaton, “A Feasibility Study of a Wearable Real-Time Notification System for Self-Awareness of body-Rocking Behavior,” IEEE Engineering in Medicine and Biology Conference (EMBC), 2019.
  4. L. Gonzalez, T. Paniagua, N. Starliper and E. Lobaton, “Signal Quality for RR Interval Prediction on Wearable Sensors,” IEEE Engineering in Medicine and Biology Conference (EMBC), 2019.
  5. N. Starliper, F. Mohammadzadeh, T. Songkakul, M. Hernandez, A. Bozkurt, E. Lobaton, “Activity-Aware Wearable System for Power-Efficient Prediction of Physiological Responses,” Sensors, 19(3), 2019, 441.
  6. F. Mohammadzadeh, C.S. Nam, and E. Lobaton, “Prediction of Physiological Response over Varying Forecast Lengths with a Wearable Health Monitoring Platform,” IEEE Engineering in Medicine and Biology Conference (EMBC), 2018.
  7. L. Gonzalez, B. Zhong, and E. Lobaton, “A Framework for Physiological Response Prediction with Joint Activity State Optimization,” IEEE Engineering in Medicine and Biology Conference (EMBC), 2018.
  8. J.P. Diaz-Paz, R.L. da Silva, B. Zhong, H. Huang, and E. Lobaton, “Visual Terrain Identification and Surface Inclination Estimation for Improving Human Locomotion with a Lower-Limb Prosthetic,” IEEE Engineering in Medicine and Biology Conference (EMBC), 2018.
  9. N. Lokare, B. Zhong and E. Lobaton, “Activity-Aware Physiological Response Prediction using Wearable Sensors,” Inventions, 2(4), 2017. [ Dataset ]
  10. B. Zhong, Z. Qin, S. Yang, J. Chen, N. Mudrick, M. Taub, R. Azevedo, E. Lobaton, “Emotion Recognition with Facial Expressions and Physiological Signals,” IEEE Symp. Series on Computational Intelligence (SSCI), 2017.
  11. N. Lokare, S. Samadi, B. Zhong, L. Gonzalez, F. Mohammadzadeh, E. Lobaton, “Energy-Efficient Activity Recognition via Multiple Time-Scale Analysis,” IEEE Symp. Series on Computational Intelligence (SSCI), 2017.
  12. N. Lokare, T. Richmond, E. Lobaton, “Robust Trajectory-based Density Estimation for Geometric Structure Recovery,” European Signal Processing Conf. (EUSIPCO), 2017.
  13. A. Dirafzoon, N. Lokare and E. Lobaton, “Action Classification from Motion Capture Data using Topological Data Analysis,” IEEE Global Conf. on Signal and Information Processing (GlobalSIP), 2016.