WO2016046789A1

WO2016046789A1 - A non-invasive rf-based breathing estimator

Info

Publication number: WO2016046789A1
Application number: PCT/IB2015/057366
Authority: WO
Inventors: Moustafa Amin Youssef; Khaled A. HARRAS; Heba ASDELNASSAR
Original assignee: Moustafa Amin Youssef
Priority date: 2014-09-25
Filing date: 2015-09-24
Publication date: 2016-03-31

Abstract

A system for nonintrusive breathing rate monitoring based on the standard WiFi equipment that works with any WiFi-enabled device without the need of any special hardware by extracting a full breathing signal and different respiratory signal information from a an input signal consisting the received signal strength (RSS) readings of the WiFi signal, thereby achieving high accuracy under different realistic deployments of the device, including on- chest and hands-free scenarios, and providing robust reading for the patient's records. In addition, the system is capable of detecting apneas experienced by the user with more accuracy.

Description

A Non-Invasive RF-based Breathing Estimator

Field of the Invention

[0001] This invention relates to wireless communication in general and to

breathing monitoring systems and WiFi based event detection systems in particular.

Background of the Invention

[0002] Respiration monitoring systems often used in hospitals typically require special contact-based devices attached to the human body, such as capnometers that are widely used in practice and that require patients to have a mask or nasal cannula constantly attached to them.

Photoplethysmography (PPG), an optical technique used to detect blood volume changes in the micro vascular tissues by illuminating the skin and measuring light absorption changes, is also used by hospitals in intensive care units via sensors attached to a patient's finger to monitor breathing and heart rates. These solutions that use special hardware may be annoying to the user and may limit the user's movement. Furthermore, these solutions are often not suitable for remote patient monitoring at home.

[0003] Along the same line, the feasibility of measuring heart rates by placing the index finger over a cell phone camera with its flash turned on or through special heart rate sensors has been demonstrated. The camera or sensor records the light absorbed by the finger tissue, and from that video, each frame is processed by splitting each pixel into RGB components which values can be used to acquire a PPG signal. Mobile phones have also been shown to measure the respiration rate of a user by analyzing the chest motion. These camera-based or sensor-based solutions share the drawbacks of being energy-intensive , requiring a certain amount of light to work properly (therefore not suitable for monitoring a person sleeping in a dark room), requiring special sensors, requiring direct contact with the user's skin, and/or raising privacy concerns.

[0004] Due to skin sensitivity and other issues that inhibit attaching sensors to the body, contact-free radio frequency (RF) respiration monitoring devices were proposed including microwave Doppler radars, ultra- wideband (UWB) radars and ISM-based systems. These systems can provide high accuracy in respiration rate detection due to the special frequency band used and/or dense devices deployment. However, their main drawbacks are limited range of the high frequency devices used as well as their high cost.

[0005] It is therefore desirable to have a non-invasive means of measuring the respiratory activity of a patient, such as breath rate and detecting apnea events, using general or specialized hardware devices which do not require direct contact with the user.

Summary of the Invention

[0006] To overcome the shortcomings of the prior art and to provide respiration monitoring systems that do not require special contact-based devices, the presented invention provides a solution for non-invasive RF-based breathing monitoring.

[0007] The present invention monitors the breathing of a person by exploiting fluctuations in the signal strength of standard WiFi signals caused by the breathing of the person using a hardware device capable of receiving a WiFi signal, which, in the preferred embodiment, can be an off-the-shelf mobile device such as a smartphone. The received WiFi signal contains a dominant periodic component which is introduced in the received WiFi signal as a result of the volume change in the chest/lungs when a person is inhaling and exhaling. The present invention leverages this component of the received WiFi signal to detect respiratory activity. The WiFi received signal strength ("RSS") can be analyzed to extract different useful information about the person's breathing pattern. The present invention works best when the hardware device is held on the person's chest or, in the hands-free mode of operation, where the person is away from the hardware device but in the line-of-sight between a WiFi transmitter (for example, a WiFi router, access point or other transmitter) and a receiver (for example, any hardware device capable of receiving WiFi signals).

The architecture of the system includes four major modules: the

Breathing Signal Extractor 102, the Stable Breathing Rate Extractor 104, the Apnea Detector 106, and the Realtime Visualizer 108. All modules can be run on the user device and the results can be offloaded to the cloud or to a local user device (e.g. a nearby laptop).

The process starts when the user activates the breathing estimation process through the system graphic user interface ("GUI") on the device. This process may be implemented, for example, as an "app" running on a smartphone device. This trigger-based activation helps in reducing the system's energy consumption, as the process is not running all the time, as well as partially reducing the effect of interference. The data acquisition module on the user's device then begins collecting the WiFi RSS in realtime.

The RSS is first processed by the Breathing Signal Extractor module 104 that filters the noise from the input signal and extracts the breathing signal from sliding windows over the input RSS stream. The

instantaneous breathing rate is also extracted in realtime.

The extracted breathing signal is then passed to the Stable Breathing Rate Extractor module 104 that filters outliers and provides a more stable signal reading (e.g. for the patient's chart) by fusing the different estimates over successive windows.

The Apnea Detector module 106 applies further de-ionizing techniques to the breathing signal and accordingly checks for the absence of the breathing pattern.

Finally, the Realtime Visualization module 108 combines the output of the different modules in a user friendly visual output and raises alarms when an apnea is detected.

Different information can be dispatched to different user devices. For example, the full information (full breathing signal, instantaneous rate and apnea alarm) can be displayed on the user's nearby laptop or sent over the Internet for remote monitoring by a health-care facility or a close family member.

Alternatively, a single time reading can be displayed on the user's device.

[0016] Fig. 1 is a block diagram showing the architecture of the present

invention.

[0017] Fig. 2(a) is a graph of an unprocessed received WiFi signal strength. Fig

2(b) shows the signal after processing by the present invention, showing extraction of the embedded breathing signal from the noisy signal strength.

[0018] Fig. 3 shows a Fast Fourier Transform ("FFT") of the raw WiFi signal in

Figure 1.

[0019] Figs. 4(a) and (b) show different techniques for mean removal for

handling a sudden change in the breathing activity

[0020] Figs. 5(a)-(c) show the processing of the signal using an a-trimmed mean filter effect for detection of outliers.

[0021] Figs. 6(a) and (b) show the detection of Biot's respiration and Apneustic

Respiration.

[0022] Referring to Fig 1, the current invention comprises the following four components: Breathing Signal Extractor module 102, Breathing Rate Extractor module 104, Apnea Detector module 106, and Realtime Visualization module 108.

[0023] Breathing Signal Extractor 102 - The breathing signal extractor module

102 extracts the breathing signal from the received noisy WiFi RSS values. Referring to Fig. 3, the frequency spectrum of the WiFi signal corresponding captured during the data acquisition process has a strong component close to the actual breathing rate. In this example, the person has a breathing rate of approximately 18 breaths per minute (bpm) (i.e. 0.3 Hz).

[0024] The present invention leverages this observation to estimate the breathing rate. To obtain the breathing rate, the first step is obtaining the frequency spectrum by applying a Fast Fourier Transform (FFT) in sub-module 102(a) to a sliding window of the WiFi raw signal strength of length n samples taken over W seconds. A band pass filter is then applied by sub- module 102(b) to limit the frequencies to those of the normal human breathing rates, shown as frequency range 302 in Fig. 3. The band pass cut-off frequencies are set to 0.1 to 0.5 Hz, which correspond to a range for the breathing rate from a minimum of 6 bpm to a maximum of 30 bpm.

[0025] The breathing rate (r^") is finally estimated as the frequency with the maximum magnitude in the human breathing rate range. More formally: = argmax |FFT(x_L..„), ¾„< r < r_max Eq. 1

[0026] where r_min is the minimum human respiration rate, r_max is the maximum human respiration rate, and x\..._n are the RSS values in the current sliding window. This produces an instantaneous estimated rate which is passed to the next system modules for further processing.

[0027] To reconstruct the breathing signal, another noise reduction step is

applied by trimming all frequencies with low energy, i.e. whose amplitude falls below 25% of the amplitude of the dominant breathing frequency, shown by line 304 in Fig. 3.

Finally, referring to Fig. 3(b), an inverse FFT operation is performed to obtain the time domain breathing signal, which is passed to the visualization module 108.

Sudden actions or movements or an interference with the line of sight between the WiFi transmitter and the WiFi Receiver may cause a sudden change in the RSS and hence in the breathing signal. A simple mean (i.e. DC component) removal does not solve this issue, as shown in Fig. 4(a), making the effect of the sudden transition appear in multiple overlapping FFT windows. To reduce this effect, a within-window local mean removal technique is applied. Specifically, from each raw RSS value in the FFT window, the mean of a shorter window (3 seconds in the preferred embodiment of the invention) centered around this value is subtracted. This local mean removal balances the signal and leads to reducing the sudden change effect, as shown in Fig. 4(b).

Stable Breathing Rate Extractor 104 - Once the breathing signal and the instantaneous breathing rate are estimated, stable breathing rate extractor module 104 attempts to increase the robustness of the breathing rate estimation based on fusing different overlapping consecutive windows. This is particularly useful in scenarios where a single stable reading is needed, for example, for logging in the patient's chart. [0031 ] The module contains two sub-modules, the Outlier Detector module 104(a) and Robustness Enhancement module 104(b).

[0032] To reduce the effect of outliers (for example, the sudden change at time t

= 52 sec in Fig. 4), an a-trimmed mean filter over a sliding window of previous instantaneous estimations is used. An α-trimmed mean filter sorts the values within the input window and then trims the highest and lowest a values. The remaining values are then averaged to obtain the smoothed filter value corresponding to the input window.

[0033] Given a window of q sorted instantaneous breathing rate estimates (rYs), such that r^"i < r^" ₂≤ ' ' '≤ r _q, the output of the filter r is given by:

where 0 < a <0.5.

[0034] For a = 0, the filter reduces to a standard moving average filter, while for a = 0.5, the filter reduces to a standard median filter. An α-trimmed mean filter has the advantage handling both impulse and Gaussian noise, as compared to mean and median filters that can handle only one of them. In the preferred embodiment of the invention, a is set to 0.25 to obtain the benefit of both types of filtering.

[0035] Referring to Fig. 5, the α-trimmed mean filter can successfully reduce the effect of the outliers as well as converge faster and more accurately to the true breathing rate compared to a standard moving average filter. Because the breathing signal is continuously changing, the present invention provides a more stable breathing rate estimate compared to the instantaneous output of the a-trimmed mean filter. This is achieved by checking whether all breathing rate estimates within the last

predetermined number of seconds are consistent within a small error margin or not. In the preferred embodiment, this predetermined number of second is fixed at 10 seconds. The system waits for the signal to become stable, introducing a slight delay for the initial robust reading. This initial delay does not affect the latency of the other modules that work in realtime.

Apnea Detector 106 - The apnea detector module 106 detects apnea, which is a cessation of the oro-nasal airflow (i.e., the absence of breathing) of at least 10 seconds in duration. The apnea is the main characteristic that marks almost all abnormal breathing patterns. During apnea, there is no movement of the muscles of inhalation and the volume of the lungs remains unchanged.

This should map to a non-changing RSS signal, modulo small changes caused by the environmental noises. The Apnea Detector module 106 works in two phases: de-noising 106(a) and apnea detection 106(b).

To detect apnea, the present invention further removes the noise from the breathing signal using wavelet de-noising in sub-module 106(a). Wavelet de-noising is a recursive filter that decomposes the input signal into multi-levels of approximation and detail coefficients. After decomposing the input signal, dynamic thresholding is applied to the wavelet detail coefficients to remove their noisy part. The inverse wavelet transform is then performed on the coefficient to reconstruct the de-noised signal. Wavelet de-noising preserves the shape of the real/desired signal better and runs efficiently in linear time in the size of the input signal.

In addition, no assumptions are made about the nature of the signal and permits discontinuities in the input signal.

An apnea consisting of a loss of the breathing signal for 10 seconds or more occurs in sub-module 106(b). To detect the loss of the breathing signal for more than 10 seconds, a sliding moving window of samples covering 10 seconds is used over the de-noised input signal. The difference between the maximum and minimum signal values within this window is compared to a threshold. If this difference is below the threshold, then there is no breathing signal and an apnea alarm is raised.

For a given window of de-noised samples y[\..m], an alarm is raised if: max(y[l..«¾]) - min(y[l..«¾]) ^< Thr Eq. 3 where Thr is the detection threshold.

To make apnea detection adapt to the dynamic environments, a dynamic threshold is used in lieu of a fixed threshold. The threshold (Thr) value is taken as a percentage (r) of the range of the last detected breathing signal. Realtime Visualization Module 108 - The Realtime Visualization module 108 combines the output of the different system modules in a user friendly and informative way. In particular, the output from the different modules is collected and displayed on the display of the user device or streamed to another user device, e.g. a laptop, to show the breathing signal, the breathing rate, as well as raising an audible and visual alarm if an apnea is detected.

Among these different outputs, the breathing signal is of specific importance, as abnormal respiratory rates and changes in respiratory rate are broad indicators of major physiological instability. Displaying the breathing waveform can help medical practitioners detect different breathing anomalies. In particular, normal ventilation is an automatic, seemingly effortless inspiratory expansion and expiratory contraction of the chest cage. This act of normal breathing has a relatively constant rate and inspiratory volume that together constitute the normal respiratory rhythm (inspiration = expiration). Abnormality may occur in the rate, rhythm, and in the effort of breathing. These properties are pretty clear in the RSS waveform after processing.

It should be noted, as would be understood by one of skill in the art, that any RF signal may be used in lieu of the WiFi signal for the purposes of this invention. For example, a cellphone signal, a Bluetooth signal, a near-field communications (NFC) signal, as well as any other RF signal now in use or contemplated in future portable hardware devices may be used for the purposes of this invention.

Regarding the user device, in the preferred embodiment of the invention, the user device is a cellular phone with smartphone capabilities, that is, the ability to download and install applications and to allow the user to interact with the applications through the user interface of the device. The user device may be, however, any hardware device capable of receiving the RF signal, for example, a laptop, a tablet or a custom device based on a hardware implementation, for example, an Arduino or a Raspberry Pi device having RF reception capabilities. In such cases, it should be noted that the software of the invention would necessarily include an additional module capable of receiving the RF signal and measuring the RSS of the signal (i.e., a data acquisition module).

Lastly, other sensors can also be used to detect other vital signals or enhance the accuracy of the breathing rate detection. For example, the signals from camera, microphone, inertial sensors (accelerometer, magnetometer, and gyroscope), among others typically found on a smartphone can be analyzed to extract the heart rate and/or fused with the RF signal to obtain better breathing rate accuracy.

In other embodiments of the invention, the exact apnea type may be detected. For example, the system is capable of detecting and

differentiating between the Biot and apneustic respiration patterns, as shown in Fig. 6(a), showing Biot's respiration and in Fig. 6(b) showing apneustic respiration.

The invention has been described in terms of specific exemplar embodiments, both hardware and software, but it should be realized by one of skill in the art that these embodiments are provided merely to illustrate the invention and that deviations from these embodiments are meant to be within the scope of the invention.

Claims

We claim:

1. A system using a hardware device capable of receiving RF signals and

providing an indication of signal strength of the received RF signal comprising: a breathing signal extractor module for extracting a breathing signal from an input signal consisting of the received signal strength readings of said RF signal; a stable breathing rate extractor module, for reducing the effect of outlying values in said breathing signal and for estimating a breathing rate based on fusing different overlapping consecutive windows and for checking whether all breathing rate estimates within a predetermined time period are consistent within a margin of error; an apnea detector module, for detecting apneas in said breathing signal; and a realtime visualization module, for combining the output of the different system modules and presenting the results to a user using the user interface of said hardware device.

2. The system of claim 1 where said breathing signal extractor module performs the further function of filtering noise from said input signal.

3. The system of claim 1 wherein said breathing signal is extracted using sliding windows over said input signal.

4. The system of claim 1 wherein said apnea detector module checks for the

absence of a breathing pattern in said breathing signal.

5. The system of claim 1 wherein said breathing signal extractor module performs the functions of: a) obtaining the frequency spectrum of said input signal within a moving window of a predetermined number of samples taken over a pre-determined period of time; b) applying a ban-pass filter to limit the frequencies to a range

corresponding to normal human breathing rates; c) estimating the instantaneous breathing rate as a function of a frequency having the maximum amplitude within said range corresponding to normal breathing rates; and d) shifting said moving window and repeating steps (a) through (d).

6. The system of claim 5 where said frequency spectrum is obtained by applying a fast Fourier transform to said input signal.

7. The system of claim 5 wherein said band pass filter uses a low frequency cutoff of approximately .1 Hz and a high frequency cutoff of approximately .5 Hz, corresponding to a breathing rate range of approximately 6 to 30 breaths per minute.

8. The system of claim 5 wherein said breathing signal can be reconstructed by

(a) applying a noise reduction filter to trim all frequencies in said frequency spectrum having an amplitude below a certain percentage of the amplitude of said frequency having the maximum amplitude; and

(b) deriving the time domain breathing signal from said frequency spectrum.

9. The system of claim 8 wherein said certain percentage is approximately 25%.

10. The system of claim 8 wherein said deriving step comprises applying an

inverse fast Fourier transform to said frequency spectrum.

11. The system of claim 5 wherein said breathing signal extractor module

performs the further function of: (a) detecting sudden changes in said input signal;

(b) taking the mean of a short window centered around said sudden deviation; and

(c) subtracting said mean from said input signal within said moving window.

12. The system of claim 5 wherein said stable breathing rate extractor module performs the functions of:

(a) applying an a-trimmed mean filter to a series of instantaneous breathing rate estimates; and

(b) averaging the filtered series of instantaneous breathing rate estimates to obtain a single stable breathing rate estimate.

13. The system of claim 12 further comprising the step of

(a) determining, prior to averaging said filtered series of breathing rate estimates, if all of said instantaneous breathing rate estimates within the last predetermined number of seconds are consistent within a margin of error; and (b) removing any instantaneous breathing rate estimates that do not fall within said small margin of error.

14. The system of claim 1 wherein said apnea detector module performs the functions of:

(a) applying a noise reduction filter to said input signal; and

(b) comparing the maximum and minimum amplitudes of signals within a second moving window of predetermined length; and

(c) determining if the difference between said maximum and minimum amplitudes falls below a predetermined threshold.

15. The system of claim 14 wherein said noise reduction filter is a wavelet de- noising filter.

16. The system of claim 14 wherein the length of said second moving window of predetermined length is approximately 10 seconds.

17. The system of claim 14 wherein said predetermined threshold is a percentage of range of the last detected breathing signal.

18. The system of claim 1 wherein said realtime visualization module performs the function of displaying outputs generated by said breathing signal extractor module, said stable breathing rate extractor module or said apnea detector module.

19. The system of claim 1 wherein said hardware device is selected from a group consisting of a laptop, a tablet, a smartphone and a custom device.

20. The system of claim 1 wherein said RF signal is a WiFi signal.