Contactless radar-based breathing monitoring of premature infants in the neonatal intensive care unit

Nearly 15 million infants are born annually before the 37th week of pregnancy, meaning that about 10% of all births worldwide are premature¹. Due to their immature organ systems and associated functions, as well as their immune system, these infants are at a higher risk of infections, chronic diseases and respiratory problems. The immaturity of breathing regulation and lungs often lead to apnea-bradycardia and respiratory distress syndromes. This is commonly followed by bronchopulmonary dysplasia in 27% of the infants born at less than 30 weeks of gestation^2,3,4,5. Consequently, further development of these premature infants has to continue ex-utero, and they usually have to spend several weeks at a Neonatal Intensive Care Unit (NICU).

During this period, continuous monitoring of their underdeveloped organs is necessary. Often, newborns are dependent on parenteral nutrition, respiratory support, and invasive diagnostic interventions which, albeit being essential for survival, may cause stress to the child. Basic vital parameters such as respiration, heart rate and oxygen saturation also need to be monitored. To this end, several sensors are directly attached to their fragile skin and connected to the monitoring systems through cables. Besides mobility restrictions, these sensors often cause skin irritation and may eventually lead to pressure necrosis^{6,7,8,9,10,11}.

In order to promote the development of premature babies, a number of efforts have been made toward non-invasive monitoring and diagnostic solutions. The use of sensors that can monitor a variety of vital signs without a cable connection, but bonded to the skin, is being investigated in^12,13. Current studies are also investigating the potential of different non-contact techniques for non-invasive diagnostics in children. Efforts are underway to detect pathological changes in body excretions by analyzing volatile organic compounds^14,15. There are approaches using optical methods to monitor pulse rate and oxygen saturation without direct skin contact and cable connection, based on e.g. dynamic light scattering¹⁶, video¹⁷ or photoplethysmography^18,19. Of high relevance for preterm infants is also the diagnosis of respiratory pathologies and classification regarding periodic breathing and apneas^20,21. This task is addressed using different non-contact techniques, which require redundant measurements of various vital signs, e.g. respiration motion, heart rate, oxygen saturation or nasal breathing^22,23,24.

The contactless monitoring of the cardiorespiratory activity neither confines nor inhibits the patient, reduces hygiene risks and does not cause any discomfort, irritation or skin damage^25,26. In this context, radars have already been proven to be a promising technology^27,28,29, being intrinsically low-power, low-cost and privacy preserving. Unlike camera-based systems^30,31, radar signals can penetrate through different materials (such as plexiglass, clothing, mattresses and blankets), and are not affected by skin pigmentation or ambient light levels. However, due to the reduced transmitted power, these signals can be easily buried in the background noise, or masked by stronger external interference, including body movements from the monitored patient³². This interference is a major challenge for accurate estimation in contactless solutions as well as for cabled devices. Specific signal processing techniques are thus needed in order to ensure reliable and robust measurements.

Recent works^33,34 have demonstrated that an ultrawideband radar can provide reliable breathing rate estimates for neonates under specific conditions. However, these investigations were limited to a single scenario, where the neonates were lying over an open-air crib, always in supine position. In addition, radar performance was evaluated only during minimal movements of the monitored patients. In this article, we take one step further by using a simpler continuous-wave (CW) radar device, and investigating premature infants under different scenarios common to the NICU routine, irrespective of the amount of movement or external interference. The specificity of the monitored patients in a real clinical setup creates several challenges which were addressed through different contributions in the proposed signal processing framework. Particularly, rather than just discarding measurements under strong interference^{33,34,35,36,37}, we present a novel random body movement mitigation technique based on the time-frequency decomposition of the recovered signal. Additionally, we propose a simple and accurate frequency estimator, which explores the harmonic structure of the breathing signal.

Problem formulation

The activity of the cardiovascular and respiratory systems causes some physical and physiological effects on the human body. The chest wall moves during the inspiration/expiration cycle as a result of the diaphragm and intercostal muscle movements. This small and periodic displacement can be detected by radar, allowing accurate estimation of the breathing rates under certain conditions. Figure 1a illustrates the basic operational principle of a CW radar. The transmitted signal propagates through the free space and reaches every object in the radar’s field-of-view, being reflected back with additional phase information regarding each object’s position. The received signal can thus be modeled as a scaled and time-shifted version of the transmitted signal, in which the phase variation over time contains valuable information regarding the scene movement. This time-varying phase \(\theta (t)\) can usually be recovered as

where \(\lambda\) is the radar operating wavelength, and d(t) represents the displacement signal which, ideally, would correspond only to the chest wall motion due to the breathing mechanism. As seen by the radar, this movement is mainly originated by the reflected points over the chest moving surface, but it may additionally include residual motion from the belly, sides and the back, depending on the patient relative position. In healthy adults, standard amplitudes for this motion range between 4 mm to 12 mm³⁸, with breathing rates varying from 5 to 25 breaths per minute (bpm)³⁹. For premature infants, these amplitudes can be smaller than 1 mm, while average breathing rates can normally reach 60 bpm⁴⁰, and go up to 80 bpm under specific conditions⁴¹.

Perfect recovery of the chest wall motion d(t) would allow precise estimation of the breathing frequency \(f_b\) by simple analysis of the movement periodicity. However, in a real clinical setup, besides unavoidable hardware imperfections, the received radar signal is usually mixed with additional reflections from the external environment, arising not only from different body movements of the monitored patient, but also from every moving object in the scene. These interfering signals are usually much stronger than those induced by the chest wall millimeter displacement, and this makes accurate recovery and subsequent estimation of the breathing frequency a challenging task. In addition, when considering premature infants, the reduced amplitudes of the chest wall motion, and the wider range of possible breathing rates pose an additional signal processing challenge in relation to previously reported research with adults.

Clinical setup and protocol

The study was performed in the Department of Pediatrics, at the Saarland University Medical Center (Homburg, Germany). Figure 2a shows a premature neonate being monitored with the conventional method. Besides the sensors attached to the chest and abdomen, and connected by cables to the central monitoring unit (for oxygen saturation, heart rate, and respiration), an additional peripheral venous catheter and a gastric tube are also necessary in this stage. The clinical setup, including the neonatal cot, the radar device, and the reference monitoring system is shown in Fig. 2b–d. The radar is certified for operation in the 24-GHz ISM (industrial, scientific and medical) band, and it was installed outside the cot, attached to a low-vibration tripod. The relative distance to the monitored infant was around 45 cm to 50 cm. Due to the radar’s inherent capabilities, no modification to the cot structure was necessary, and the plastic cover could remain closed during the measurements. In Fig. 2d, twins are sharing the same bed, with only one being monitored with the contactless method. Cobedding of twins is a common procedure in the NICU, with several studies reporting physiological benefits to the infants^42,43.

A total of 12 premature infants were included in the study. The Supplementary Table 1 shows a summary of the patient’s information. They were selected on the basis of medical opinion, and taking into account the medical safety of participating in the study. For each infant, the measurements were carried out in three different days, at noon (after feeding), over a period of 25 minutes each. Their natural position was not changed during each measurement. Besides the supine (with the chest facing the radar), prone (with the back facing the radar) and side positions, we have also investigated cobedding cases with only one infant being monitored using the contactless method. The idea was to investigate the different effects when collecting radar data from the chest/abdomen and back. Additionally, if monitoring twins is possible, and what would be a safe distance (in terms of radar interference) between them. The basic principle that guided the data collection protocol was to ensure seamless operation at the NICU. A detailed description of the patient’s protocols is shown in Supplementary Tables 2a–d, including all interventions and additional transients manually annotated during the measurements.

Signal processing background

Figure 1b shows the basic block diagram of the signal processing chain. The initial signal processing step for CW systems is commonly known as phase demodulation. It is essentially the process where the received in-phase and quadrature (I and Q) signals from the radar’s analog-to-digital converter (ADC) are combined with the aim to recover the displacement signal d(t). Among several methods, the two most used are the arctangent demodulation (AD)⁴⁴ and the complex signal demodulation (CSD)⁴⁵. While the AD enables precise recovery of the chest wall motion, it is highly sensitive to hardware calibration, and to the presence of DC offsets, noise, and external interference. The CSD is more robust to these effects, but it relies on small displacements for recovering an approximation of the breathing motion (please refer to the Methods).

Figure 3 shows examples of the recovered breathing motion from radar data, in comparison to the actual (reference) displacement acquired from the cabled device. Initially, to precisely reconstruct the chest wall displacement, we selected “clean” segments of data (no external interference), and the AD was used in both cases. While Fig. 3a depicts a normal breathing pattern obtained at supine position, Fig. 3b shows an occurrence of the Cheyne-Stokes (periodic) breathing pattern⁴⁶, with the infant at prone position. This special form of breathing is physiologically found in neonates, and is defined by a cyclic variation between hyperpnea and hypopnea^47,48,49, i.e. repetitive short cycles of pauses and breaths. Despite small differences between the recovered radar signals and the reference device, the periodic breathing movement can still be clearly identified in both cases. The small amplitudes of the chest wall motion can also be visualized, with displacements around 2 mm in supine position and 0.5 mm in prone position. These amplitudes are well below typical values for adults reported in previous research^50,51,52.

The approximated breathing motion, obtained with the CSD, is shown in Fig. 3d,e. Despite noticeable differences when compared to the AD, Fig. 3g,h show that both techniques yield signals with the same fundamental frequency, corresponding to the average breathing frequency. Given the small displacements we aim to detect, and the challenging conditions of this real clinical environment, the CSD was adopted in our solution for long-term monitoring. The harmonic structure of the breathing signal can also be visualized, with the second harmonic being clearly distinguishable. This harmonic structure can be used for improving estimation, as we will show latter. However, as depicted in Fig. 3c,f,i, under external interference and eventual ADC saturation, both demodulation methods fail to reconstruct the chest wall motion. The spectrum becomes dominated by the interference components, which will then prevent accurate breathing frequency estimation. Therefore, additional processing is necessary in order to attenuate these effects.

Random body movement mitigation

Most research on contactless vital sign monitoring with radar sensors focus on a single-person setup under ideal motionless conditions⁵³. In practical monitoring situations, the subject may often move body parts like hands, legs or torso, and even the entire body. These unwanted but unavoidable movements are usually called random body movements (RBM). The amplitude of their reflected signals is often much stronger than the millimeter-scale breathing motion, which will potentially be masked by this interference. Since spontaneous RBM are inevitable, solving this problem is fundamental to reliable vital sign detection in practical applications.

Several methods for RBM mitigation were already proposed⁵⁴, and even though specific types of movements could be effectively cancelled out, they usually require more complex systems. Most solutions rely on additional or duplicated hardware, thus suffering from practical limitations such as misalignment, synchronization, and cost^45,55,56. Another direction of research basically tries to identify segments of vital sign data with RBM, and simply discard these corrupted segments before estimation^{33,34,35,36,37}. However, depending on the processing window duration, even very short RBM will affect several seconds of good signal. Therefore, rather than simply discarding segments of data, an approach which allows useful exploitation of these episodes with moderate RBM is desired. Recent work has begun to address RBM using a single sensor and within more challenging scenarios³². Nonetheless, experimental validation is still performed under controlled situations, with RBM being emulated through predefined behavior, which results in limited interference over the desired signal.

First, let us assume that RBM are sparse, i.e. they are not frequent and, when they occur, their duration is small in relation to the observed time window. This contrasts with the constant and periodic nature of the breathing movement. Additionally, their amplitudes are usually much stronger than the standard breathing signal. These specific time and frequency features will be present in the spectrogram of the recovered signal, which can be analyzed toward identifying and possibly removing this interference. For addressing this, we will use the nonnegative matrix factorization (NMF)^57,58, a matrix decomposition technique usually employed for extracting features from a set of nonnegative data. If x(t) (Fig. 1b) is the recovered signal containing the chest wall motion and eventual RBM interference, its magnitude spectrogram \(|{{\varvec{X}}}|\) can be obtained through the Short-Time Fourier Transform (STFT) of x(t). The NMF will then decompose \(|{{\varvec{X}}}|\) as

where the matrices \({{\varvec{H}}}\) and \({{\varvec{W}}}\) contain, respectively, the associated time and frequency basis components of \(|{{\varvec{X}}}|\), with K being a predefined number of basis. In other words, \({{\varvec{W}}}\) can be seen as the set of frequency templates of \(|{{\varvec{X}}}|\), while \({{\varvec{H}}}\) contains the timing information related to the activation of these templates. If we look into the time activation matrix \({{\varvec{H}}}\), the basis components with sparse behavior and higher amplitudes will often indicate the epochs when the RBM interference is present. Despite the unpredictable frequency spectrum, which will eventually overlap with breathing frequencies, the RBM distinct time behavior can be captured by the NMF time activation bases \({{\varvec{H}}}\), whereas the corresponding bases in \({{\varvec{W}}}\) will retain its frequency content. This allows additional flexibility for filtering the RBM interference when compared to standard spectral analysis methods. We can thus reconstruct the filtered spectrogram \(\hat{|{{\varvec{X}}}|}\), by simply adding back all the \({{\varvec{w}}}_i {{\varvec{h}}}^{\mathrm{T}}_i\) matrices, except for the ones containing the interfering components.

Figure 4a shows a 60-s processing window for illustrative purposes, where the recovered signal x(t) (after CSD) is corrupted by segments of RBM, with its normalized spectrogram \(|{{\varvec{X}}}|\) in Fig. 4b. In the case of the CSD, the spectrogram is calculated based on the complex samples of the recovered signal x(t), and therefore considers both I and Q channels simultaneously. The NMF decomposition into \(K=11\) frequency (\({{\varvec{W}}}\)) and time basis (\({{\varvec{H}}}\)) components is depicted in Fig. 4c,d, where each color represents a pair of basis components, with the frequency content in Fig. 4c, and the corresponding time activation in Fig. 4d. It can be seen that (please refer to the green and blue bases for instance), due to its random nature, the RBM interference has frequency components spread over the entire spectrum, overlapping with the breathing frequency region. While a variety of frequencies can be visualized in \({{\varvec{W}}}\), the sparse and strong bases corresponding to RBM can be clearly identified in \({{\varvec{H}}}\) (please refer to the Methods). Removing the selected bases allows the reconstruction of the filtered spectrogram in Fig. 4e, where the breathing frequency variation over time (around 45 bpm) is now evident. After the inverse STFT, the RBM filtered time signal is depicted in Fig. 4f. Finally, Fig. 4g shows the bandpass spectrum of both the original and the RBM filtered signals. The corresponding detected values are highlighted respectively with the blue and red markers. The dashed black line shows the reference value for the average breathing frequency associated with this processing window. Because of the strong RBM interference, the maximum value of the original spectrum would indicate an erroneous breathing frequency of 52.1 bpm, very distant from the true value of 42 bpm. After RBM filtering with the NMF, the modified spectrum indicates a closer value of 42.4 bpm, where the estimation error would be only 0.4 bpm.

Breathing rate estimation

Different models have already been proposed for representing the back-and-forth breathing movement d(t), from simple sinusoidal approximations^59,60, to more complicated patterns as described in^38,61. The breathing movement is a complex phenomenon which involves different patterns of motion, not only from the chest wall surface, but also from the belly, shoulders and back^62,63. Therefore, it is difficult to identify time-domain models that fully characterize it in a robust way, for every subject and monitoring scenario. However, due to the inherent periodic nature of breathing, any function representing this movement can eventually be decomposed into Fourier terms, containing the fundamental frequency and harmonics that correspond to the breathing rates we aim to estimate. Hence, the displacement signal can be modeled as a sum of harmonically related complex sinusoids, having frequencies that are integer multiples of the fundamental breathing frequency. To better exploit this harmonic structure, in this section we propose a simple and accurate Nonlinear Least Squares (NLS) estimator⁶⁴, which is asymptotically efficient for large processing windows, even in colored noise scenarios⁶⁵.

Initially, for removing any residual DC values, and possible high frequency noise components, the RBM-filtered displacement signal \({\hat{x}}(t)\) is further filtered using a bandpass Kaiser window (\(\beta =6.5\)), from 0.3 Hz to 3 Hz (\(18-180\) bpm). This corresponds to the physiological range of breathing frequencies, also including possible harmonics. The bandpass filtered signal \({\hat{d}}(t)\) will ideally be an accurate approximation of the true chest wall motion d(t) (Fig. 1), and can finally be used for breathing frequency estimation.

Before estimation, for improving the signal-to-noise ratio (SNR)^66,67,68, we calculate the autocorrelation function r(t) of the bandpass filtered signal. The estimation is first performed in time domain, directly over the autocorrelated signal. An initial (coarse) estimation is obtained through a peak detection algorithm, where the time distance between peaks provides an estimation of the time between each breath. Eventually, detected peaks can be excluded if the distance to its neighbors correspond to a frequency outside the expected physiological range. The initial breathing frequency is thus calculated as the inverse of the time between selected peaks, averaged over the entire processing window. This initial estimation will be used to simplify the NLS algorithm.

The following step is to calculate the NLS frequency estimates \({\hat{\omega }}\), which are obtained by maximizing the similarity between \({\hat{d}}(t)\) and the displacement signal model in (12). Under certain conditions (please refer to the Methods), the solution to this problem (the resulting cost function in (17)) can be efficiently implemented using a Fast Fourier Transform (FFT) and a linear grid search algorithm⁶⁹, where the estimator reduces to a summation of the breathing harmonics over the power spectral density of \({\hat{d}}(t)\). The initial time domain estimation is used for limiting the search range, thus avoiding stronger low-frequency components which may still be present in real data. This strategy also reduces the computational effort to perform the grid search.