Non-Contact Automatic Respiratory Rate Monitoring for Newborns Using Digital Camera Technology and Deep Learning
Huda Ali Hashim
(Login required)
Middle Technical University, Baghdad, Iraq
Paper #9164 received 3 Sep 2024; revised manuscript received 27 Nov 2024; accepted for publication 28 Nov 2024; published online 26 Dec 2024.
DOI: 10.18287/JBPE24.10.040317
Abstract
In healthcare settings, particularly for newborns in the Neonatal Intensive Care Unit (NICU), continuous and accurate monitoring of respiratory rates (RR) is crucial, yet current methods often involve manual region selection, which limits their effectiveness. To address this challenge, we developed a novel approach using a low-cost digital camera for non-contact RR monitoring of ten newborns. This study leverages deep learning for automated region of interest (ROI) selection in the face and chest areas, coupled with signal decomposition techniques to reduce noise artifacts. A graphical user interface (GUI) system was also introduced to facilitate real-time RR monitoring and visualization of results. The experimental outcomes demonstrate the system’s high efficacy, with the object detector achieving precision rates of 98.24% for the face region and 96.61% for the chest region. Additionally, the system recorded a low average mean absolute error (MAE) of 1.11 breaths per minute for the face and 1.03 breaths per minute for the chest, corroborated by reference ECG monitor readings. This method offers a cost-effective, non-contact, and deployable solution suitable for both clinical and home health monitoring applications.
Keywords
Full Text:
PDFReferences
1. X. He, R. A. Goubran, and X. P. Liu, “Wrist pulse measurement and analysis using eulerian video magnification,” 2016 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), IEEE, 41−44 (2016).
2. A. G. Shabeeb, H. A. Hashim, and S. K. Gharghan, “Heart disease classification based on combination of PCA/ANFIS model,” Research on Biomedical Engineering 40(3–4), 609–625 (2024).
3. H. A. Hashim, N. M. Ahmed, and A. G. Shabeeb, “Infant heart rate estimation based on non-contact UV photoplethysmography,” Indonesian Journal of Electrical Engineering and Computer Science 31(1), 180–188 (2023).
4. M. Chen, Q. Zhu, H. Zhang, M. Wu, and Q. Wang, “Respiratory rate estimation from face videos,” in 2019 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI), IEEE, 1–4 (2019).
5. D. Kolosov, V. Kelefouras, P. Kourtessis, and I. Mporas, “Contactless Camera-Based Heart Rate and Respiratory Rate Monitoring Using AI on Hardware,” Sensors 23(9), 4550 (2023).
6. L. Scalise, N. Bernacchia, I. Ercoli, and P. Marchionni, “Heart rate measurement in neonatal patients using a webcamera,” in 2012 IEEE International Symposium on Medical Measurements and Applications Proceedings, IEEE, 1–4 (2012).
7. J. Brieva, E. Moya-Albor, O. Rivas-Scott, and H. Ponce, “Non-contact breathing rate monitoring system based on a Hermite video magnification technique,” in 14th International Symposium on Medical Information Processing and Analysis 10975, 19–25 (2018).
8. A. Al-Naji, K. Gibson, S.-H. Lee, and J. Chahl, “Monitoring of cardiorespiratory signal: Principles of remote measurements and review of methods,” IEEE Access 5, 15776–15790 (2017).
9. A. Al-Naji, A. G. Perera, and J. Chahl, “Remote monitoring of cardiorespiratory signals from a hovering unmanned aerial vehicle,” Biomedical Engineering Online 16, 1–20 (2017).
10. W. Verkruysse, L. O. Svaasand, and J. S. Nelson, “Remote plethysmographic imaging using ambient light,” Optics Express 16(26), 21434–21445 (2008).
11. L. Tarassenko, M. Villarroel, A. Guazzi, J. Jorge, D. Clifton, and C. Pugh, “Non-contact video-based vital sign monitoring using ambient light and auto-regressive models,” Physiological Measurement 35(5), 807 (2014).
12. Y. Zeng, D. Yu, X. Song, Q. Wang, L. Pan, H. Lu, and W. Wang, “Camera-based cardiorespiratory monitoring of preterm infants in NICU,” IEEE Transactions on Instrumentation and Measurement 73, 1–13 (2024).
13. G. O. Ganfure, “Using video stream for continuous monitoring of breathing rate for general setting,” Signal, Image and Video Processing 13(7), 1395–1403 (2019).
14. J. H. Klaessens, M. Van Den Born, A. Van Der Veen, J. Sikkens-van De Kraats, F. A. Van Den Dungen, and R. M. Verdaasdonk, “Development of a baby friendly non-contact method for measuring vital signs: First results of clinical measurements in an open incubator at a neonatal intensive care unit,” SPIE Proceedings 8935, 89351P (2014).
15. M.-Z. Poh, D. J. McDuff, and R. W. Picard, “Non-contact, automated cardiac pulse measurements using video imaging and blind source separation,” Optics Express 18(10), 10762–10774 (2010).
16. Y. J. Chen, L.-C. Lin, S.-T. Yang, K.-S. Hwang, C.-T. Liao, and W.-H. Ho, “High-reliability non-contact photoplethysmography imaging for newborn care by a generative artificial intelligence,” IEEE Access 11, 90801–90810 (2023).
17. G. Boccignone, A. D’Amelio, O. Ghezzi, G. Grossi, and R. Lanzarotti, “An evaluation of non-contact photoplethysmography-based methods for remote respiratory rate estimation,” Sensors 23(7), 3387 (2023).
18. L. Svoboda, J. Sperrhake, M. Nisser, L. Taphorn, and H. Proquitté, “Contactless assessment of heart rate in neonates within a clinical environment using imaging photoplethysmography,” Frontiers in Pediatrics 12, 1383120 (2024).
19. A. Al-Naji, K. Gibson, S.-H. Lee, and J. Chahl, “Real time apnoea monitoring of children using the Microsoft Kinect sensor: a pilot study,” Sensors 17(2), 286 (2017).
20. L. A. M. Aarts, V. Jeanne, J. P. Cleary, C. Lieber, J. S. Nelson, S. Bambang Oetomo, and W. Verkruysse, “Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit—A pilot study,” Early Human Development 89(12), 943–948 (2013).
21. W. Wang, S. Stuijk, and G. De Haan, “A novel algorithm for remote photoplethysmography: Spatial subspace rotation,” IEEE Transactions on Biomedical Engineering 63(9), 1974–1984 (2015).
22. H. Hashim, S. Mohammed, and S. Gharghan, “Accurate localization of elderly people based on neural and wireless sensor networks,” ARPN Journal of Engineering and Applied Sciences 14(11), 3777–3789 (2019).
23. A. Al-Naji, J. Chahl, “Remote respiratory monitoring system based on developing motion magnification technique,” Biomedical Signal Processing and Control 29, 1–10 (2016).
24. F. K. Lucy, K. T. Suha, S. T. Dipty, M. S. I. Wadud, and M. A. Kadir, “Video based non-contact monitoring of respiratory rate and chest indrawing in children with pneumonia,” Physiological Measurement 42(10), 105017 (2021).
25. B. Olmi, C. Manfredi, L. Frassineti, C. Dani, S. Lori, G. Bertini, S. Gabbanini, and A. Lanata, “Aggregate channel features for newborn face detection in neonatal intensive care units,” in 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 455–458 (2022).
26. F.-T.-Z. Khanam, A. G. Perera, A. Al-Naji, K. Gibson, and J. Chahl, “Non-contact automatic vital signs monitoring of infants in a neonatal intensive care unit based on neural networks,” Journal of Imaging 7(8), 122 (2021).
27. Z. Jiang, L. Zhao, S. Li, and Y. Jia, “Real-time object detection method based on improved YOLOv4-tiny,” arXiv preprint arXiv:2011.04244 (2020).
28. Y. Yuliza, R. Muwardi, M. Rhozaly, L. Lenni, M. Yunita, and G. Yehezkiel, “Fast human recognition system on real-time camera,” Jurnal Ilmiah Teknik Elektro Komputer dan Informatika 9(4), 895–903 (2023).
29. X. Li, J. Pan, F. Xie, J. Zeng, Q. Li, X. Huang, D. Liu, and X. Wang, “Fast and accurate green pepper detection in complex backgrounds via an improved Yolov4-tiny model,” Computers and Electronics in Agriculture 191, 106503 (2021).
30. R. Zhu, L. Zhao, “Research and application of face mask detection algorithm based on YOLOV4-tiny,” Academic Journal of Computing & Information Science 5(1), 64–69 (2022).
31. S. Hraybi, M. Rizk, “Examining YOLO for real-time face-mask detection,” IET Conference Proceedings 2021(11), 571–575 (2022).
32. A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, “Yolov4: Optimal speed and accuracy of object detection,” arXiv preprint arXiv:2004.10934 (2020).
33. S. Chaichulee, M. Villarroel, J. Jorge, C. Arteta, G. Green, K. McCormick, A. Zisserman, and L. Tarassenko, “Multi-task convolutional neural network for patient detection and skin segmentation in continuous non-contact vital sign monitoring,” in 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), 266–272 (2017).
34. J. Jorge, M. Villarroel, S. Chaichulee, K. McCormick, and L. Tarassenko, “Data fusion for improved camera-based detection of respiration in neonates,” Optical Diagnostics and Sensing XVIII: Toward Point-of-Care Diagnostics 10501, 215–224 (2018).
35. A. Y. Abdulrazzak, S. L. Mohammed, A. Al-Naji, and J. Chahl, “Real-time jaundice detection in neonates based on machine learning models,” BioMedInformatics 4(1), 623–637 (2024).
36. Z. Wu, N. E. Huang, “Ensemble empirical mode decomposition: a noise-assisted data analysis method,” Advances in Adaptive Data Analysis 1(01), 1–41 (2009).
37. W. Chen, D. McDuff, “Deepphys: Video-based physiological measurement using convolutional attention networks,” Proceedings of the European Conference on Computer Vision (ECCV), 349–365 (2018).
38. D. McDuff, S. Gontarek, and R. W. Picard, “Improvements in remote cardiopulmonary measurement using a five band digital camera,” IEEE Transactions on Biomedical Engineering 61(10), 2593–2601 (2014).
39. J. Brieva, H. Ponce, and E. Moya-Albor, “A contactless respiratory rate estimation method using a hermite magnification technique and convolutional neural networks,” Applied Sciences 10(2), 607 (2020).
40. A. Al-Naji, J. Chahl, “Remote optical cardiopulmonary signal extraction with noise artifact removal, multiple subject detection & long-distance,” IEEE Access (6), 11573−11595 (2018).
Сontact
34 Moskovskoe shosse, Samara, 443086, Russian Federation
Email: j-bpe@ssau.ru
Phone: +7-846-267-4550
© 2014-2025 J-BPE