Introduction

In the intensive care unit (ICU), spontaneous breathing trial (SBT) has been a standard procedure to assess whether a patient can be weaned from a mechanical ventilator (MV)1. However, 10–20% of patients experience extubation failure, even if they have passed the SBT2. Compared to patients who successfully undergo extubation, those who experience extubation failure may face a higher risk of in-hospital mortality3. Although most physiological parameters are monitored during SBT, the insight on movement patterns of respiratory muscles are lacking4. The diaphragm, a dome-shaped respiratory muscle, serves a crucial function for approximately 80% of the respiratory volume in mammals5. In ICU, approximately 20–25% of patients on MV face challenges with weaning, and a significant portion of these cases can be attributed to diaphragmatic dysfunction6. Additionally, cough strength, which is one of the indicators of weaning readiness2, for cleaning secretions, is not assessed objectively in routine before and after SBT1,7.

Ultrasound has become an insightful technique to investigate diaphragmatic movement and function8. Compared with other methods, ultrasound provides a potentially noninvasive, convenient, and radiation-free means of diaphragmatic visualization9,10, and has similar accuracy to most other modalities in diaphragm measurement11. Portable ultrasound demonstrates the potential for monitoring various organs12, including the diaphragm. Motion mode (M-mode)13,14 or M-mode-like15,16 ultrasound has been employed to observe the positions of the diaphragm and pleura using bioadhesive13,14 or wearable single transducer15,16. However, due to the limited visual field of M-mode, these devices have drawbacks in terms of spatial orientation and poor repeatability11,17.

Considering the irregular profile and motion of the diaphragm, brightness mode (B-mode) ultrasound provides a more comprehensive perspective for monitoring the diaphragm, offering improved spatial resolution. Recently, researchers have begun to explore various segmentation18,19,20 and tracking methods12 for quantitatively analyzing diaphragmatic function, including thickness, thickening fraction, strain, and excursion. Threshold methods require manual or semi-automatic pre-setting of a region of interest (ROI) and involve time-consuming preprocessing steps. To solve this, ultrasound image tracking algorithms, such as normalized cross-correlation and optical flow, have been used for diaphragm tracking19,21,22. Although the ensemble threshold method can select tracking points semi-automatically22, these traditional tracking methods still cannot achieve fully automated processing. Moreover, the measurement of B-mode ultrasound is susceptible to the orientation of the ultrasound beam, as well as to compressive maneuvers during ultrasound examination and manual image analysis23.

In this proof-of-concept study, we aim to develop an easy-to-use multi-view ultrasound system that automatically locates and quantifies B-mode ultrasound of the diaphragm from both intercostal and subcostal views. Importantly, we focus on diaphragm monitoring and cough strength estimation, which can be used for evaluating weaning readiness and predicting weaning outcome. To achieve this, the proposed system consists of two wearable angle-free probes (Fig. 1a, b), a diaphragm tracking algorithm (Fig. 1c), and a downstream algorithm (Fig. 1d). The wearable probe allows for rapid setup and automatic diaphragm positioning by controlling the transducer orientation, thus meeting the demands for low obstruction and high applicability in bedside monitoring within the ICU. To provide real-time quantification of diaphragm morphology and movement, we propose a loop-twice diaphragm tracking (LTDT) algorithm, which is automated and applicable for multiple ultrasound views. It demonstrates satisfactory anti-motion capabilities in segmenting the diaphragm during violent movement. Finally, we observe real-time patterns during diaphragm contraction in healthy participants, demonstrating the validity and feasibility of bedside monitoring and cough strength estimation using the proposed ultrasound system.

Fig. 1: Overview of the multi-view ultrasound system for diaphragm monitoring.
Fig. 1: Overview of the multi-view ultrasound system for diaphragm monitoring.
Full size image

A Probes placement. Linear-array (green) and curvilinear (blue) probes were positioned at the intercostal space of the anterior axillary line and the anterior subcostal region through the acoustic window of the liver, respectively. The markers of the probes are consistent in the following figures. B Exploded view of the angle-free probes. The hinge connected the probe body (a) to the linear-array (b) or curvilinear transducer (c). C Diaphragm positioning and tracking algorithm. \(I\): Current B-mode images; \({M}_{0}\): Initial masks; \(C\): Confidence scores; \(\widehat{M}\): Segmentation mask; Memory features were represented by memory key \(k\) and value \(v\). D Diaphragm monitoring during normal respiration and cough, and algorithm for downstream task, e.g., cough strength estimation. \(f\): Features extracted from monitoring signals; \(t\): Task outputs.

Methods

Multi-view ultrasound system with angle-free probes

The hinge of an angle-free probe connects the transducer to the probe body (see Fig. 1b and Supplementary Fig. S1a). An ultrasound processing module (Beijing Sonoptek Co., Ltd, China), lithium battery, and interface board (Arduino Nano, Arduino LLC, Italy) were vertically stacked in a 3D-printed case of the probe body. Linear (Bandwidth: 7.5 MHz ± 35%) and curvilinear transducers (Bandwidth: 3.2 MHz ± 35%) mounted in custom cases were attached to the probe body for intercostal and subcostal diaphragm imaging, respectively. The choice of probe placement was based on availability in a supine position and imaging difficulties6. The built-in motor (15 rpm speed, 9 kg cm torque) is controlled to rotate the transducer through gear outside the case. The probe assembly had a total mass of 137.9 g for the linear probe, and 165.6 g for the curvilinear probe. An elastic strap attached to the top allows the angle-free probe to be quickly secured in the target position, ensuring low obstruction and interference when wearing loose clothes (see Supplementary Movie S1). The internal battery enables the probe to operate wirelessly for more than two hours, facilitating potential bedside monitoring. The B-mode ultrasound images from multiple views were transmitted to the terminal through Wi-Fi in real-time.

Sample size and participants

A priori analysis conducted using G*Power software (Heinrich-Heine-Universität Düsseldorf, Germany) estimated a required sample size of 46 participants for the Pearson correlation, based on a Cohen’s effect size of r = 0.496 from a previous study24. This analysis aimed for a statistical power of 0.95 and a two-sided significance level of 0.05 within a bivariate correlation design. Considering the testing of the machine learning models, additional test sets needed to be prepared to avoid potential information leakage before use, leading to untrustworthy results. To account for potential dropouts and facilitate an 80/20 dataset split, the final target sample size was increased to sixty participants. Finally, sixty healthy participants (29 males and 31 females; age = 26.7 ± 3.6 years; weight = 62.7 ± 13.2 kg; height = 168.6 ± 7.6 cm, presented as mean ± standard deviation) from the Hong Kong Polytechnic University were recruited. All of them signed informed consent and participated in the complete experimental process, which included a study of validity and cough strength estimation. The experiments received approval from the Human Subjects Ethics Subcommittee at The Hong Kong Polytechnic University (HSEARS40408014).

Experimental setup and data collection

In this proof-of-concept study, we evaluated the accuracy and feasibility of the proposed ultrasound system (Fig. 1a). Before experiments, we synchronized the proposed system by timing samples and eliminating delays among devices (Supplementary Methods and Fig. S2). In the first step, we conducted a validity study on the ultrasound system by comparing its measurements with those from a commercially available ultrasound scanner, Aixplorer’s (SuperSonic Imagine, France) (Fig. 2b). Participants were instructed to lie down in a supine position with their right arms raised behind their heads. During the validity study, the respiration belt (Go Direct, Vernier, USA) provided participants with visual feedback (Fig. 2a), which ensured that breathing amplitude in each measurement was consistent. The sonographer used clinical ultrasound probes (L15-4 and XC6-1, SuperSonic Imagine, France) to collect B-mode images of the right hemidiaphragm during three breathing cycles. The linear-array transducer was placed within the 8th to 10th intercostal space (ICS) of the anterior axillary line. The curvilinear probe was placed at the anterior subcostal region through an acoustic window of the liver11. Probe positioning was guided by the requirement for clear and consistent visualization of the diaphragm throughout \(\ge \! 5\) quiet breathing cycles and during maximal coughing efforts. The data collection process was then repeated with the proposed system. The wearable probes were secured to the body with an adjustable elastic strap to ensure appropriate probe-skin pressure while maintaining optimal skin contact (Supplementary Methods). After the sonographer quickly fixes the probes to the correct placements in three steps (Supplementary Fig. S1 and Movie S1), the system automatically locates the clear diaphragm view and begins data acquisition. We then examined the feasibility of diaphragm monitoring and cough strength estimation for potential ICU applications using the multi-view system (Fig. 2b). With the wearable probes on and the respiration belt off, participants were asked to perform 2 coughs with maximal effort. Cough peak flow (CPF), recognized as a reliable measure of cough strength25, was assessed with a peak flow meter (SMPF-2S, Sonmol, China). B-mode ultrasound from multiple views were simultaneously collected and processed in real time.

Fig. 2: Experimental setup and ablation studies on LTDT.
Fig. 2: Experimental setup and ablation studies on LTDT.
Full size image

A Illustration of experimental setup. A participant wore the multi-view ultrasound system and a reseparation belt. Visual feedback (Supplementary Fig. s2b) was provided to ensure consistent breathing amplitude in the validity study. B Illustration of experimental procedure. C Quantitative results of ablation studies on loop-twice diaphragm tracking (LTDT). We performed LTDT inference on the test set and conducted ablation studies by disabling either data augmentation or the global memory module in LTDT. ϑ & Φ, the average of the Jaccard index and boundary accuracy was considered the primary metric for evaluation. D Qualitative results of ablation studies. We selected two frames from each of the six example sequences in the test set. Three of the sequences were from intercostal images of the diaphragm collected using a wearable probe. The remaining sequences contained one of each type. Seq Sequence.

Diaphragm positioning algorithm

In the autotuning process, the diaphragm positioning algorithm and angle-free probe formed a closed loop (Supplementary Fig. S3a). A trained recurrent neural network (RNN) was provided with states of the adjacent and current frame, which are stored in the state bank. The output actions controlled the transducer orientation through the motor and captured the B-mode ultrasound frame. Then, the current frame is matched with template images of the diaphragm to calculate mutual information, which would be passed to an updated state bank. Fourteen image sequences (Dataset I in Supplementary Fig. S4a) were acquired using angle-free probes to train the positioning algorithm. The NeuroEvolution of Augmenting Topologies26 was utilized to evolve the genome of the RNN model. The implementation details can be found in the previous work27.

To achieve fully automatic processing, a multi-task U-net was trained on Dataset Ⅱ to segment and provide the confidence score of the current segmentation (Supplementary Fig. S3b). Once the confidence score hits the target, the transducer achieves the optimal orientation, and the tracking begins. The current frame \(I\) and initial mask \({M}_{0}\) initialized the LTDT algorithm. The multi-task model involves one more encoder after the decoder in the U-net to form an additional branch27.

Loop-twice diaphragm tracking

We developed the LTDT, a custom one-for-all Python algorithm to track diaphragm morphology in different views. LTDT loops twice to track the target based on memory features stored in the global memory module (Fig. 1c). Global memory was updated during the first inference and frozen at the end of it. This strategy reduces excessive dependence on the first frame and enhances the anti-motion interference ability of the model. Given the ultrasound image sequence \({I}_{{{\rm{H}}}\times {{\rm{W}}}\times 3}\) and the initial diaphragm mask \({M}_{0,{{\rm{H}}}\times {{\rm{W}}}\times 1}\), LTDT provided segmentation masks for the whole sequences \({\widehat{M}}_{{{\rm{H}}}\times {{\rm{W}}}\times 3}\).

We prepared two datasets for tracking model training. The dataset Ⅱ (Supplementary Fig. S4b) was anonymized and comprised sequential ultrasound images of muscles and annotations of muscle borders, e.g., pleura, peritoneum, or aponeurosis. A total of 1251 images from 35 sequences were collected using different devices targeting the diaphragm, gastrocnemius, tibialis anterior, and adductor pollicis muscle to enhance the dataset’s variability. Dataset III (Supplementary Fig. S4c) was obtained by selecting 2495 key frames from 271 ultrasound videos collected in this study. The selection of key frames prioritizes image diversity. Hence, the key frames are dense in large motion (e.g., expulsion stage of cough) and sparse in mild motion (e.g., end of the expiration). The images were annotated by an experienced sonographer. We used the subject-level holdout method to split the dataset, resulting in 48 training individuals (including 1966 images from 212 sequences) and 12 test individuals (including 529 images from 69 sequences). Samples from the same individual did not appear in both the training and test sets simultaneously. Subsequent training and statistical analysis of features were performed on the training set to ensure that the test set was not used before evaluation. We selected the ROI by estimating the active region to prevent interference from redundant information in the ultrasound images (Supplementary Fig. S5b). By summing the differences between adjacent frames, the region with the most intense movement can be identified, and one-dimensional thresholds were established in both directions to obtain the ROI.

LTDT model consisted of three trainable convolutional networks: an encoder, a decoder, and a value encoder (see Supplementary Fig. S5c). We first pretrained the network in dataset II with large diversity. This pretraining process warmed up the network using ultrasound modalities with hyperechoic borders. We adopted the open-source implementation of Space-Time Correspondence Network (STCN)28 without modifications. STCN serves as a minimalistic form of matching networks for tracking. Then, we performed the main training on the diaphragm sequence dataset (training set of dataset III), which was augmented using bidirectional inference29 (Supplementary Fig. S5a). This semi-supervised method produced pseudo-labels from STCN inference, resulting in densely annotated ultrasound sequences. We allocated 10% of the training dataset as a validation set to select hyperparameters. To minimize training time, five previous frames were randomly chosen and stored in the memory bank for each query during the training process. We employed bootstrapped cross-entropy loss and Dice loss with equal weighting. We used the AdamW optimizer30 with a learning rate of 1e-5 and applied 20k iterations for the main training, using a batch size of 16.

To evaluate the LTDT algorithm and how pseudo-annotations and global memory affected the segmentation performance, we performed LTDT inference on the test set of diaphragm sequences (Supplementary Fig. S4c) and conducted ablation studies on different settings by disabling data augmentation or the global memory modules in LTDT. The open-source implementation of STCN28 was used as a baseline when both settings were disabled. For quantitative analysis, we first calculated standard metrics without the first frame by comparing with the ground truth \({M}_{{{\rm{H}}}\times {{\rm{W}}}\times 1}\): Jaccard index ϑ, boundary accuracy Φ, and their average ϑ & Φ, where higher values are better31. ϑ & Φ was considered as the primary metric due to its comprehensiveness and common use in video segmentation tasks32.

Quantification of diaphragm

In the resulting binary mask \(\widehat{M}\) from LTDT, the opening transform was used to remove the burrs and the falsely segmented contours. In the intercostal view, the diaphragm presented two high-echo regions: the pleural and peritoneal layers surrounding it. The subcostal view showed a high-echo arc with up-and-down motion. Sonomyography (SMG) refers to the quantitative analysis of muscle function based on architectural change in ultrasound images33. When the diaphragm moves, its shape, size, and position change, which can be quantified to SMG signals for functional analysis. However, these changes are not typical image transformations, such as affine, projective, rotation, and scaling. Therefore, we used various representation methods to quantify the thickness and excursion of the diaphragm (Supplementary Fig. S5d). In intercostal view, assuming that the pleural and peritoneal layers appear as two low-curvature morphologies, we fitted the points on the two contours by two third-order polynomials. The thickness was calculated by averaging the vertical distance between two polynomials (Supplementary Fig. S6a). In subcostal view, we adapted a heuristic method34 to find the principle moving direction of the diaphragm based on principal component analysis (PCA), and excursion was obtained by mapping inter-frame distance of mean points to the principal direction (Supplementary Fig. S6b). The PCA-based method tolerates the incompleteness34 of the diaphragm and provides vector excursions that are better for trend observation.

In the respiration experiment and the validity study, diaphragm thickness (\({T}_{{\mathrm{ex}}}\)) and excursion were calculated at the end of expiration. The thickening fraction is defined as \(({T}_{{\mathrm{in}}}-{T}_{{\mathrm{ex}}})\cdot 100 \% /{T}_{{\mathrm{ex}}}\), where \({T}_{{\rm{in}}}\) represents the thickness at the end of inspiration. In the cough experiment, we defined the onset of cough \({t}_{{\mathrm{on}}}\), end-expulsion time \({t}_{{\mathrm{ex}}}\), and end-compression time \({t}_{{\mathrm{com}}}\) by locating valleys and peaks of SMG signals (Supplementary Methods). We primarily focused on eleven features extracted from SMG signals, including inspiration time, expulsion time, thickness at end-compression, thickness at the onset of cough, thickening fraction, average thickening speed, peak thickening speed, relative thickening speed, passive excursion during expulsion, average velocity, and peak velocity. The definitions of these features will be introduced alongside the SMG pattern of cough in the subsequent section.

Cough strength estimation

To examine potential ICU applications using the multi-view system, we aimed to develop a downstream algorithm for cough strength assessment suitable for all medical situations. A pipeline for machine learning model selection was implemented within the training set through five-fold cross-validation, which involved feature selection, hyperparameter tuning, and regressor selection. The correlation analysis of eleven features extracted from the diaphragmatic SMG was used as a criterion for pre-selection. Features with p < 0.05 were considered effective and selected. Seven common machine learning regressors, namely Ridge, Lasso, support vector regressor, k-nearest neighbor, random forest (RF), gradient boosting decision tree (GBDT), and AdaBoost, were employed as candidates through the scikit-learn implementation35. Optuna36, an optimization framework, was adopted to select optimal hyperparameters for each regressor in a large search space. We performed a total of 35k trials on 1 to 3 hyperparameters for each regressor (see details in Supplementary Table 1). After model selection, the optimal regressor with the lowest cross-validated root-mean-square error (RMSE) was finally evaluated on the test set.

Statistics and reproducibility

Statistical analyses were performed to evaluate correlation relationships among diaphragm thickness, thickening fraction, and excursion measured by Aixplorer’s and our multi-view ultrasound system. Eleven features extracted from diaphragmatic SMG during two coughs and simultaneously measured CPF were used to analyze correlations. The performance of regression models constructed with different feature combinations was evaluated by analyzing the relationship between true and estimated CPF. Pearson correlation coefficients (r) were used to estimate the correlation. Bland-Altman plots were utilized to illustrate the agreement, with differences calculated by subtracting our method’s measurements or estimations from the reference measurements or ground truth values. For pairwise comparisons on RMSEs for different regression models, we used one-way repeated measures analysis of variance (ANOVA). All the above analyses were conducted by the Statistical Package for Social Sciences (SPSS, IBM Corporation, USA), with a significance level set at 0.05.

Results

Loop-twice diaphragm tracking

Across all test sequences (Fig. 2c, Supplementary Table 2), our LTDT model achieved an overall ϑ and Φ of 0.792, ϑ of 0.736, and Φ of 0.848, which was the highest among all ablation settings, especially in intercostal images of the diaphragm from wearable probe. In these images with relatively poor image quality and more intensive diaphragm motion, we noted LTDT achieved better results (ϑ & Φ of 0.771) compared to others (ϑ and Φ of 0.713 and 0.743). The distribution of ϑ and Φ for each sequence showed a right-shifting trend, indicating that with the enabling modules in LTDT, the cases with poor performance gradually diminished, leading to better overall results of the model. In our model (blue), there are very few cases where ϑ and Φ is less than 0.7. In comparison studies, our LTDT outperformed other state-of-the-art models in semi-automated video object segmentation tasks (Supplementary Table 3 and Figs. S7, S8).

We selected two images from each of the six example sequences in the test set to demonstrate qualitative results (Fig. 2d). Three of the sequences were from intercostal images of the diaphragm collected using the wearable probe. The remaining sequences contain one of each type. Example images were generally taken from the start and end keyframes of the motion. In the majority of frames with different views, LTDT correctly segments the diaphragm region. In subcostal view (Seq 5, Seq 6), the diaphragm presented a simple arc morphology with high contrast, which could be easily tracked in detail by our model (ϑ and Φ of 0.810 and 0.855). Even the baseline model achieved comparable results in these cases (ϑ and Φ of 0.795 and 0.854). In contrast, the diaphragm from intercostal view presented two echogenic layers, which are prone to segmentation errors from other linear structures, such as motion artifact (frame 6 of seq 1), connective tissue, vessels11 (frame 10 of seq 1, seq 4), as well as an unidentified line in the middle37 (frame 2 of seq 3). The weakening of the boundary intensity in keyframes (seq 2) also increases the difficulty of tracking. Our method effectively reduced cases of false segmentation and mis-segmentation while handling details better (frame 5 of seq 3).

Diaphragm monitoring and its normal sonomyographic pattern

To verify the feasibility and validity of the system for respiratory monitoring, we performed qualitative and quantitative comparisons between diaphragmatic B-mode ultrasound from our system and medical ultrasound devices. Segmented sequences from participants (n = 60) were quantified to obtain diaphragmatic SMGs, i.e., thickness from intercostal view, and excursion from subcostal view. Data from two representative participants showed a correlation between SMGs during normal respiration (Fig. 3a). The inspiratory process is accompanied by diaphragmatic contraction and movement toward the peritoneum, resulting in an increase in muscle thickness (the distance between the two membranes) and increased excursion. During expiration, the diaphragm relaxes and returns to its original state, showing a decrease in thickness and a cranial displacement. We observed that the SMG signal of thickness showed a unimodal curve (participant a) or a multimodal curve (participant b), depending on the shape of the intercostal diaphragm (see “Discussion”).

Fig. 3: Diaphragm monitoring during respiration and validity study.
Fig. 3: Diaphragm monitoring during respiration and validity study.
Full size image

A Example B-mode ultrasound and SMGs from two participants using Aixplorer’s during respiration. Since the images were not collected simultaneously, the SMGs from the two views are synchronized according to the respiration belt. B Example B-mode ultrasound and SMGs from two participants using wearable probe during respiration. The images were captured synchronously by multi-view system. In A, B, the respiration cycle is defined by the valley of the respiration belt signal, from the beginning of the inhalation to the end of the exhalation. The line and shaded area represent mean ± standard deviation of SMG (n = 3 cycles). The gray and red markers indicate the beginning and end of inspiration. The quintic Savitzky-Golay filter with a window length of eleven was used to smooth the signals for better illustration. The white markers inside the ultrasound images indicate the distance representation of the target layers or the expected direction of displacement. C–E Three SMG features measured by wearable probes vs. Aixplorer’s with Bland-Altman plots across all participants (n = 60). In scatter plots, the distribution graphs in the top and right margins correspond to the parameters on the horizontal and vertical axes, respectively. The two-sided Pearson correlation analysis was used to assess if there is a significant correlation (Thickness: p < 0.001; Thickening fraction: p < 0.001; Excursion: p < 0.001). F Outcome SMG features from different genders across all participants (n = 60).

In comparison, the results from the wearable probe (Fig. 3b) were similar to those from the Aixplorer’s probe. Quantitatively, we studied the three common measurements of diaphragm6. Diaphragmatic thickness (r = 0.973, p < 0.001), thickening fraction (r = 0.862, p < 0.001), and excursion (r = 0.893, p < 0.001) from the two devices were highly correlated, and the linearity of regression models was strong (Fig. 3c–e). Although there is considerable deviation in single measurements due to potential confounding factors, the Bland-Altman plots indicated a small mean difference between the probes (Thickness: −0.03 mm; Thickening fraction: 1.82%; Excursion: 0.11 mm). This suggests that despite variability in individual measurements, our wearable probes provide a marginal error in measurement, as evidenced by the overall results from multiple measurements. Such performance suggests that the multi-view system can precisely capture and quantify diaphragm movements in synchronized views, as well as identify specific patterns of diaphragmatic movements among different participants. We concluded SMG features from different genders across all participants (Fig. 3f), where thickness = 2.1 ± 0.6 mm (Male: 2.2 ± 0.6 mm, Female: 2.0 ± 0.5 mm), thickening fraction = 17.5 ± 7.8 % (Male: 16.9 ± 7.7 %, Female: 18.2 ± 7.8 %), excursion = 13.7 ± 4.4 mm (Male: 14.8 ± 4.8 mm, Female: 12.5 ± 3.4 mm). Furthermore, we compare measurements with those reported in other studies38,39,40 (Supplementary Table 3). The results indicated that our findings regarding gender differences were consistent with those observed in previous studies. The male’s diaphragm showed smaller contractions41 but produced greater excursion38. Additionally, we performed a single-participant study to investigate the feasibility of bedside monitoring. We observed that our LTDT algorithm and multi-view system can work normally for long-term when freezing the global memory module, and diaphragmatic SMGs showed a periodic pattern (Fig. 1d, Supplementary Movie S2).

Sonomyographic pattern of cough

To investigate the further application in movement recognition and estimation of cough strength, we collected B-mode images of 60 participants during two coughs. Data from a representative participant showed unique and synchronized motion patterns of cough across SMGs (Fig. 4a). Similar to the inspiratory phase in normal respiration, we noticed that a typical cough began with contraction of the diaphragm (First blue interval), resulting in thickening and caudal displacement. The difference is that the deep inspiratory phase is followed by a compression phase (Gray interval) with glottic closure42. During this phase, compression by the abdominal muscle led to a significant increase in intrapleural and intra-abdominal pressure, resulting in a sudden high-pressure gradient between the mouth and the thoracic cavity. The diaphragm tends to maintain its current position (presenting as small excursion), yet the enormous intracavitary pressure forces it to switch its contraction pattern (see “Discussion”), which causes it to deform and results in a violent change in thickness43. After the glottis opens, the pressure gradient generates an expulsive flow from the airway to the mouth (Red interval). The diaphragm relaxes and passively moves cranially, followed by a return to its original shape and position (Last blue mark). Such discernible results demonstrate that our technique provides a means of real-time analysis to explore the insightful patterns of respiratory movements. Moreover, SMGs enabled subsequent feature extraction and downstream applications.

Fig. 4: Sonomyographic pattern of cough.
Fig. 4: Sonomyographic pattern of cough.
Full size image

A Example B-mode ultrasound and SMGs from one participant during a cough. The images and SMGs were obtained synchronously from multi-view system. The quintic Savitzky–Golay filter with a window length of eleven was used to smooth the signals for better illustration. The white markers inside the ultrasound images indicate the distance representation of the target layers or the direction of displacement. B Correlation analysis of thickening fraction (\({\mathrm{TF}}\), Corr = 0.69, p < 0.001). C Correlation analysis of peak thickening speed (\({{\mathrm{TS}}}_{{\mathrm{peak}}}\), Corr = 0.67, p < 0.001). D Correlation analysis of average thickening speed (\({{\mathrm{TS}}}_{{\mathrm{ave}}}\), Corr = 0.50, p < 0.001). E Correlation analysis of thickness at end-compression (\({T}_{{\mathrm{com}}}\), Corr = 0.50, p < 0.001). F Correlation analysis of relative thickening speed (\({{\mathrm{TS}}}_{{\mathrm{rel}}}\), Corr = 0.48, p < 0.001). G Correlation analysis of passive excursion (\({\mathrm{PExc}}\), Corr = 0.36, p < 0.001). H Correlation analysis of peak velocity (\({V}_{{\mathrm{peak}}}\), Corr = 0.32, p < 0.010). I Correlation analysis of average velocity (\({V}_{{\mathrm{ave}}}\), Corr = 0.31, p < 0.010). Eight SMG features that have significant correlations with cough peak flow (p < 0.050) are shown in the figure and ordered by correlation coefficient. Features from some unmeasurable samples were dropout due to unsatisfactory ultrasound imaging quality. Corr: Pearson correlation coefficients. For the scatter plots in B–I, the distribution graphs in the top and right margins correspond to the parameters on the horizontal and vertical axes, respectively. The center lines and shaded regions represent regression lines calculated by the least squared method, along with its 95% confidence intervals. The two-sided Pearson correlation analysis was used across all participants (n = 60) to assess if there is a significant correlation.

Cough strength estimation

To evaluate cough strength estimation using the multi-view system, CPF measured during cough was used as the indicator of cough strength25 to conduct a correlation analysis (n = 96 from two coughs of 48 training individuals). Given the thickness \(T(t)\) and excursion \({\mathrm{Exc}}(t)\) of the diaphragm, we primarily focused on eleven SMG features extracted from SMG signals, including inspiration time (\({\Delta t}_{{\mathrm{in}}}={t}_{{\mathrm{com}}}-{t}_{{\mathrm{on}}}\)) from the onset of cough to end-compression, expulsion time (\({\Delta t}_{{\mathrm{ex}}}={t}_{{\mathrm{ex}}}-{t}_{{\mathrm{com}}}\)) from end-compression to end-expulsion, thickness at end-compression (\({T}_{{\mathrm{com}}}\)), thickness at the onset of cough (\({T}_{{\mathrm{on}}}\)), thickening fraction (\({{\mathrm{TF}}}=({T}_{{\mathrm{com}}}-{T}_{{\mathrm{on}}})\cdot 100 \% /{T}_{{\mathrm{on}}}\)), average thickening speed (\({{{\mathrm{TS}}}}_{{\mathrm{ave}}}=({T}_{{\mathrm{com}}}-{T}_{{\mathrm{on}}})/{\Delta t}_{{\mathrm{in}}}\)), peak thickening speed (\({{\mathrm{TS}}}_{{\mathrm{peak}}}=\mathop{\max }_{{t}_{{\mathrm{on}}} < t\le {t}_{{\mathrm{com}}}}\frac{\Delta T(t)}{\Delta t}\)), relative thickening speed (\({{\mathrm{TS}}}_{{\mathrm{rel}}}={\mathrm{TF}}/\Delta {t}_{{\mathrm{in}}}\)), passive excursion during expulsion (\({\mathrm{PExc}}={\mathrm{Exc}}\left({t}_{{\mathrm{com}}}\right)-{\mathrm{Exc}}({t}_{{\mathrm{ex}}})\)), average velocity (\({V}_{{\mathrm{ave}}}={{\mathrm{PExc}}}/{\Delta t}_{{\mathrm{ex}}}\)), and peak velocity (\({V}_{{\mathrm{peak}}}=\mathop{\max }_{{t}_{{\mathrm{com}}} < t\le {t}_{{\mathrm{ex}}}}\frac{{\mathrm{\varDelta Exc}}(t)}{\Delta t}\)). For the first time, we considered the active thickness changes from intercostal view during the inspiration and compression phases, which reflected the maximum intensity of diaphragm contraction, especially against high intracavity pressure. The results showed that 5 of the 6 thickness related features had a significant correlation with cough strength (p < 0.050, Fig. 4b–f), and the correlations were higher than those of the excursion-related features. \({\mathrm{TF}}\) (p < 0.001, Corr = 0.69) and \({{\mathrm{TS}}}_{{\mathrm{peak}}}\) (p < 0.001, Corr = 0.67), as relative values, showed the highest correlation among all, while the correlation of absolute values is relatively low and even non-significant (\({T}_{{\mathrm{on}}}\): p = 0.791, Corr = −0.03; \({T}_{{\mathrm{com}}}\): p < 0.001, Corr = 0.50). Regarding the excursion obtained from subcostal view, hypothesizing that diaphragmatic cranial motion results from expulsion24, we focused on passive movement during expulsion that previously has been reported as an important predictor of CPF44. All three excursion-related features showed significant correlations with CPF (p < 0.050, Fig. 4g–i). However, compared to thickness, features reflecting passive movement were less efficient and even showed no significant correlation in the male population (\({\mathrm{PExc}}\): p = 0.102; \({V}_{{\mathrm{peak}}}\): p = 0.311; \({V}_{{\mathrm{ave}}}\): p = 0.381). We observed a small number of outliers in the correlation analysis at high CPF and feature values. We also noticed that the average values of all features were greater for males than females (see top distribution of Fig. 4b–i, Supplementary Table 5), even in the \({TF}\) that was not consistent with the results in normal respiration. When asked to cough with maximum effort, males showed stronger diaphragmatic contractions and explosive power to produce airflow through airway (CPF = 279.1 ± 67.2 L/min). Lastly, the two time-dependent features (\({\Delta t}_{{\mathrm{in}}}\): p = 0.670, Corr = 0.04; \(\Delta {t}_{{\mathrm{ex}}}\): p = 0.498, Corr = −0.07) showed no significant correlation with CPF and would be removed in the subsequent model selection.

Next, based on participants’ characteristics (i.e., gender, age, weight, height, and BMI) and eight SMG features significantly correlated with CPF, we aimed to develop a downstream algorithm for cough strength estimation suitable for various medical situations. The t-SNE method45 and Shapley Additive exPlanations (SHAP)46 were utilized to visualize and interpret the performance of SMG features from model-free and model-based perspectives (Fig. 5a, Supplementary Fig. S9). All selected features were decomposed into two dimensions, showing their correlation with cough strength. It’s noted that the majority of samples with the strongest (Deep red) and weakest (Deep blue) cough strength were roughly concentrated in the lower-right and upper-left regions, while most samples of intermediate strength (Light color) around or in between. This recognized distribution pattern demonstrates the effectiveness of selected feature sets in the downstream task. Following the model selection pipeline, the GBDT with tuned hyperparameters (Supplementary Table 5) achieved the lowest error in cross-validation (n = 5, RMSE = 44.7 ± 8.9 L/min), and was selected. We evaluated the model on the test set and obtained a generalized model with RMSEs of 49.2 ± 35.8 L/min and r of 0.73 across all coughs (n = 24, Fig. 5b). We also observed that the model performed similarly in both the validation and test sets, indicating no overfitting and satisfactory robustness of the model. The Bland-Altman plot revealed notable deviations in individual estimates; however, overall, our algorithm showed only slight overestimation across all samples (mean = −4.0 L/min). Most cases were densely distributed within the confidence limit and close to the mean value. We explored the influence of features on the model using SHAP values (Supplementary Fig. S7a). Ordered by the sum of SHAP values for all samples, thickness-related features played a dominant role in the cough strength estimation, as expected from the correlation analysis. In addition, \({\mathrm{PExc}}\) (ranked third) also contributed by conveying information from subcostal view.

Fig. 5: Cough strength estimation.
Fig. 5: Cough strength estimation.
Full size image

A t-SNE feature space using all selected features, only intercostal features, and only subcostal features. The different colors of sample points represent a stronger (red) and weaker (blue) cough strength. B Scatter plot and Bland–Altman plot showing difference between true and estimated CPF in all participants (n = 120). C RMSEs using different feature subsets across coughs in test individuals (n = 24). One-way ANOVA showed no significant main effect (p = 0.942, F2, 61 = 0.060).

Lastly, as proof of real-world applicability when one of the views is unavailable in the system, we conducted two additional model selections using only intercostal features and using only subcostal features. Two AdaBoost models with distinct hyperparameters were developed and evaluated (Intercostal model: n_estimator = 332, learning_rate = 2.54; Subcostal model: n_estimator = 500, learning_rate = 2.71). Although model performance was slightly degraded when using the intercostal feature subset (RMSE = 53.0 ± 38.1 L/min) and subcostal feature subset (RMSE = 51.9 ± 33.2 L/min), we observed no significant difference in RMSEs among them (p = 0.942, F2,61 = 0.060) (Fig. 5c). The results demonstrated the availability of the multi-view system under non-ideal conditions. However, in the feature space, we observed significant changes in the distribution of the samples. We obtained relatively poor models with r values of 0.53 and 0.47 across all coughs (n = 24) and observed more outliers in scatter plots (Supplementary Fig. S10).

Discussion

In this proof-of-concept study, we developed a multi-view ultrasound system designed to automatically locate and quantify the diaphragm in intercostal and subcostal views. We first demonstrate the effectiveness of the proposed LTDT algorithm in tracking the diaphragm using multi-view B-mode ultrasound, particularly under great motion disturbance. We demonstrate the feasibility of the multi-view system for diaphragm monitoring and cough strength estimation. In comparison to other modalities, diaphragmatic SMG enables a deeper investigation into the underlying mechanism of respiratory movements.

Our system, featuring an elastic strap attached at the top (Supplementary Fig. S1b), allows for the angle-free probe to be quickly secured in the target position through three simple steps (Supplementary Fig. S1c). This configuration ensures appropriate probe-skin pressure while maintaining image quality and accommodating various patient conditions. In future study, we may integrate a force sensor within the strap to monitor the applied force, and an automatic adjustment could be implemented to modify the strap length based on the detected force.

Our LTDT algorithm outperformed other widely used video segmentation models in comparison studies. Ablation studies indicate that this superior performance results from strategies involving semi-supervised data augmentation and robust anti-motion interference through the global memory module. The data augmentation significantly mitigated overfitting. Over-reliance on information from the previous frame may lead to poor segmentation performance, especially at the onset of the violent motion. Global memory eliminates reliance on the first frame during initializing, allowing second-stage inference to provide robust processing of boundary edges and motion artifacts. However, fat or food stagnation in the stomach can obscure the complete morphology of the diaphragm through acoustic window of the liver, potentially hindering the segmentation task (Supplementary Fig. S8). Such imaging challenges may lead to the unavailability of one ultrasound view, a situation likely to be more common in ICU patients with complex medical conditions. This stresses the necessity of investigating the downstream algorithm using SMG features from a single view.

In the Bland-Altman plots from the validity study, we observed a small mean difference despite considerable variation in some features. This variation arises from measurement errors associated with several confounding factors: (1) While we used a respiration belt to minimize differences in breath amplitude during each measurement, errors may still exist; (2) Although a professional sonographer and secured belt placement ensured consistent probe positioning, ideal quality and complete consistency in visualization cannot be guaranteed due to anatomical variations47, such as diaphragmatic slip and crura48, which can lead to potential measurement discrepancies. However, using anatomical landmarks and maintaining a supine position can minimize overall data variability and enhance reproducibility49,50. Specifically, commencing at the 9th ICS as an initial reference, adjustments (cranial or caudal) were made within the 8th–10th ICS range according to individual differences and ultrasound image characteristics to improve image quality and reduce bias. Furthermore, our study involved multiple measurements, which help mitigate errors from single measurements. This underscores the importance of collecting multiple respiratory cycles and coughs in practical applications, whether for monitoring or cough measurement, to calculate an average for more reliable results.

We further showed that our multi-view system could be employed for real-time monitoring of various respiratory movements. The absolute values of some measurements obtained by our system appear inconsistent with the previous reports. Variations in subject positions, ultrasound modes, and calculation methods may result in different measurements (listed in Supplementary Table 4). Regarding thickness, some studies51 have found that diaphragm thickness may increase due to the gravitational pull of abdominal and thoracic viscera when the subject is upright. Our results (2.1 ± 0.3 mm) are consistent with studies conducted in a supine position6,52, showing lengthened and thickened layers of the resting diaphragm. Moreover, measurement methods utilized in different ultrasound modes can lead to relatively large discrepancies in measured values (Supplementary Fig. S6a). First, the selection of reference points varies among studies. Second, the targeted area is important. Considering that tracking or segmenting a single point in speckled ultrasound can be challenging, we utilized the regional average method to facilitate target area selection6. This approach affects the thickening fraction when the diaphragm exhibits uneven regional changes. We observed that the thickening of the diaphragm in the central intercostal area tended to be more intense and rapid, whereas for the diaphragm below the ribs, this was limited, resulting in a relatively small thickening fraction calculated by regional average (23.9 ± 9.4%)6. This may explain the unique SMG pattern observed with a multimodal curve. In some cases, the central intercostal diaphragm rebounded from the large deformation, causing the boundary of the diaphragm below the ribs to bend upward (participant b of Fig. 3b). This resulted in the overall average thickness rising to form a second peak, after which the layers gradually returned to parallel as thickness decreased. Using our quantification method, we can investigate the uneven changes in different areas of the diaphragm. Regarding diaphragmatic excursion (Supplementary Fig. S6b), the diaphragmatic movement direction is non-linear and variable with the phase, and the selection of interrogation lines in M-mode ultrasound is prone to subjective influence, resulting in a limited observation area, low repeatability, and significant bias among studies11,17. Therefore, several studies20,22 have attempted to quantify the overall movement of the diaphragm using the semi-automated B-mode method. In our study, we employed PCA to analyze diaphragm displacement in the principal direction at each time point. By using our method, the measured value (13.7 ± 4.4 mm) is consistent with the mainstream method, and the real-time analysis of SMG (Figs. 3, 4) can better reflects the overall motion while accommodating diaphragm incompleteness34.

Our system provides detailed SMG patterns during different phases of a cough, reflecting diaphragm motions that other bio-signals cannot capture. To our knowledge, few studies have detailed the phases of a cough using ultrasound techniques53,54. An effective cough occurs when increased intrathoracic pressure leads to dynamic compression of the airways55. Hence, the pressure gradient generated during the compression phase is crucial56; however, it is often confused with the respiratory phase in sonography24,44,57, both of which are accompanied by increases in excursion and thickness. However, we observed that the diaphragm activity is distinct in different phases, as reflected in the SMG pattern. Specifically, during the inspiratory phase, diaphragm contractions allow for shortening, yielding diaphragmatic displacement, which can be a concentric contraction. The force generated during this phase is primarily responsible for inhalation and is typically less than the muscle’s maximum force58. In contrast, during the compression phase, the diaphragm may need to contract isometrically to withstand intense pressure from the expiratory muscles while being held at a fixed length. This results in a dramatic increase in diaphragm thickness (Fig. 4a) and may exert maximum force58, playing a role in maintaining the pressure gradient necessary for generating an effective cough. A similar phenomenon has been reported in animal experiments using EMGs43, where significant bursts were observed in the costal and sternal regions at the end of inspiration.

To describe the relationship between diaphragmatic SMG and cough strength, we extracted several SMG features representing active thickening and passive excursion of the diaphragm, most of which showed a significant correlation with CPF. Specifically, thickness-related features from intercostal views exhibited stronger correlations (higher r value in Fig. 4b–f) and contributed more to the regression models (higher SHAP value in Supplementary Fig. S9), due to the nature of cough dynamics and fewer cases with poor image quality. While the outliers in correlation analysis may affect the results, their impact is limited due to their small number. Their presence may be related to smaller sample sizes within certain ranges. The active thickness change of the diaphragm reflects its maximum effort during the compression phase; however, related research is limited and warrants further attention in future studies. It’s important to note that this study did not extract features for the compression phase separately, as its patterns are not consistently recognized among individuals due to its short duration (approximately 200 ms). Sonomechanomyography techniques59,60 with frame rates exceeding 1000 frames per second could be utilized to sample more frames from the compression phase in future studies.

The results of cough strength evaluation confirm the system’s feasibility for real-world applications. Notable deviations in the Bland-Altman plots (Fig. 5b) indicate a need for larger sample sizes in both model training and cough estimation to mitigate confounding factors. Future studies should collect data from a diverse range of patient states, measurement locations, and probe orientations to enhance sample variability and improve model generalization. Additionally, we recommend conducting multiple assessments in real-world applications to ensure reliable outcomes, considering the minimal overall mean differences between true CPF and estimated CPF (Fig. 5b).

In summary, our research successfully demonstrated the feasibility of analyzing diaphragm activity during distinct respiratory movements and implementing downstream tasks, including CPF estimation using our multi-view ultrasound system and insightful SMG techniques, illustrating their potential as a common setting in the ICU for various applications during and after SBT. We developed customized software, RespiraLab, integrated with LTDT and downstream algorithm for real-time diaphragm monitoring and cough strength estimation (Supplementary Movie S2). Further clinical evaluation in ICU patients will be necessary to verify the full potential of this system.