Introduction

Cardiovascular diseases (CVDs) are the leading cause of morbidity and mortality worldwide. According to the World Health Organization (WHO), there were an estimated 17.9 million deaths from CVDs, accounting for 32% of all global deaths in 2019. Furthermore, this number is expected to grow to approximately 23.6 million by 2030. Over three-quarters of CVD deaths occur in low- and middle-income countries due to the inaccessibility of primary healthcare, despite the fact that these diseases are preventable through early detection and treatment1. Therefore, CVDs pose an enormous burden on society.

Photoplethysmography (PPG), which enables non-invasive monitoring of the cardiovascular system, along with mobile health (mHealth) technology that utilizes mobile computing and communication technologies (e.g., mobile phones, wearable sensors) for health services and information2, can be a promising solution to alleviate the burden by enabling ubiquitous monitoring. This technology uses smartphone applications or wearable devices that are deeply ingrained in everyday life.

However, current PPG sensors are built with rigid inorganic semiconductor-based light-emitting diodes (LEDs) and silicon photodiodes (PDs), which are relatively large and measure a few square millimeters in size. This limitation prevents the integration of multiple sensing channels in standalone medical devices or wearables (e.g., smartwatches and rings), restricting measurements to a single point. Consequently, the detectable biomarkers—such as heart rate, stress, oxygen saturation, and blood pressure (BP) at the wrist, finger, or earlobe—are insufficient for comprehensive cardiovascular health monitoring (cardiovascular disease (CVD), high-accuracy BP, etc.) in a ubiquitous manner. Moreover, the widespread adoption of smart wearable technologies in clinical practice is still hindered by challenges such as device accuracy, clinical validation, a lack of standardized regulatory policies, and patient privacy concerns3. In particular, the deficiencies in accuracy and clinical validation are critical issues that need to be solved first.

To address the aforementioned issues, we implemented a multifunctional mobile cardiovascular health monitor (MCHM) within the indispensable device—smartphones—by utilizing a massive array of miniaturized PPG sensors integrated with a sophisticated organic light-emitting diode (OLED) display (referred to as “Sensor OLED”). This approach leverages advanced ultrafine display manufacturing processes. As a result, we achieved distinctive capabilities, including concurrent multi-point PPG sensing (MPS), high-resolution dynamic image sensing (HDIS), and user-interactive sensing (UIS). This allows the device to extend new functions to monitor cardiovascular health based on the user’s physiological signals obtained from a simple measurement (placing fingertips on the display).

A high accuracy of 90% in CVD screening was achieved. BP sensing, utilizing a state-of-the-art machine learning (ML) algorithm, demonstrated improved performance compared to the existing medical-grade wearable algorithm when using a single finger. Further enhancements in accuracy and usability were observed when PPG signals from two fingers were used. These results were validated through pilot studies. The real-time user-interactive PPG sensing also achieved high PPG quality (SNR ≥ 20 dB) with a reduced failure rate. HDIS assists in identifying finger blood vessels and assessing blood flow dynamics, allowing for the diagnosis of blood vessels and circulation quality from the captured abundant high-resolution images. This capability has enabled new sensing methods (spatial and spatial-temporal sensing) that we believe are beneficial, although they require further validation in the future. Most importantly, these features are not currently supported by conventional wearable devices. As a result, true ubiquity in self-monitoring for cardiovascular health has been achieved with a user-friendly approach, alongside medical device-level accuracy, by utilizing smartphones embedded with Sensor OLED technology—without the need for additional wearable or attachable standalone PPG sensors. This advancement could transform smartphones into ubiquitous health monitors and digital therapeutics, significantly impacting diagnostics and treatment. It offers substantial benefits for individuals seeking healthier lives and alleviates the burden on society, particularly in low-resource settings where access to medical and wearable devices is limited.

This paper builds upon our previous works4,5. We introduce the following components: (1) the concept of the proposed multifunctional mobile cardiovascular health monitor (MCHM), and (2) the implementation of a multi-point PPG sensing OLED display with an enhanced signal-to-noise ratio (SNR). Next, we will present the application aspects categorized by the distinctive sensing capabilities and associated functions. For the MPS, we will discuss (3) the cardiovascular health monitor (CHM), (4) the blood pressure monitor (BPM), and (5) the finger blood vessel and blood flow dynamics sensing enabled by the HDIS. Lastly, we will provide a brief introduction to the UIS and its application aspects. These topics were not covered in our previous reports.

Results

Concept of the multifunctional mobile cardiovascular health monitor (MCHM)

PPG is a simple yet effective optical sensing technique that non-invasively measures blood volume changes in the microvascular bed of tissue. It provides valuable information related to the cardiovascular system and is widely used in medical and wearable devices. However, the relatively large current PPG sensors hinder multi-channel integration, thereby limiting biomarker detection for comprehensive cardiovascular health monitoring in ubiquitous interaction settings.

Displays are one of the most effective user-interactive mediums in electronic systems, providing instantaneous and impactful visual feedback. Given their user-friendly nature, massively integrating optical sensors into OLED displays can create tremendous synergy for interactive and ubiquitous health monitoring. Consequently, we proposed the Sensor OLED display, using OLED technology as both the display and light source, with organic photodiodes (OPDs) serving as photodetectors. This integration enables sophisticated UIS, capable of MPS and HDIS from a compact, pixilated design (as small as ~µm² per pixel) without increasing thickness.

As a result, smartphones equipped with Sensor OLED display can broaden their applications to include mobile cardiovascular health monitoring and digital therapeutics, completely eliminating the hassle of carrying standalone devices or separate sensors, as well as reducing the need for frequent hospital visits.

The Sensor OLED offers several clear advantages over conventional PPG sensors:

  1. 1.

    Multi-Point Sensing (MPS): This feature allows distinct multi-site PPG sensing across the large display area, enabling high-accuracy cardiovascular disease (CVD) monitoring by meticulously comparing pulses from the left and right fingers. It also enhances BP sensing using PPG pulses from two fingers.

  2. 2.

    High-Resolution Dynamic Image Sensing (HDIS): This capability enables blood vessel imaging and blood flow dynamics assessment, which could be valuable for diagnosing conditions such as systemic sclerosis (Raynaud’s phenomenon)6, peripheral arteriolar lesions7, and Buerger’s disease8, where vascular Doppler detectors are currently employed. Additionally, HDIS has potential applications in authentication and anti-spoofing (Supplementary Figs. 1 and 2).

  3. 3.

    User-Interactive Sensing (UIS): It provides a convenient and interactive experience, allowing users to simply place a finger on the smartphone display. It offers real-time signal quality feedback during sensing, helping to improve signal quality by reducing motion artifacts—the primary source of noise in wearable PPG sensors—and significantly lowering the sensing failure rate (Supplementary Fig. 3). Furthermore, UIS creates a single smartphone-based biofeedback device, reducing the need for separate devices or augmented reality (AR)/virtual reality (VR) systems with additional PPG sensors (Supplementary Fig. 4c).

Therefore, the Sensor OLED-based smartphone could integrate expensive medical devices such as CVD monitors, dual-cuff blood pressure monitors (BPMs), and vascular Doppler detectors, as well as standalone devices like atrial fibrillation (AF) monitors, biofeedback devices, and heart rate monitors (HRMs). Ultimately, it could evolve into a new type of mobile cardiovascular health monitor, as illustrated in Fig. 1a, enabling mHealth by further leveraging the Internet of Things (IoT) and artificial intelligence (AI), potentially revolutionizing diagnostics and treatment. Figure 1b highlights the technical impacts, showing that complex cardiovascular disease screenings, which are currently only performed in hospital settings, and dual-cuff BPMs, which provide more precise BP monitoring9,10, can be replaced by a single smartphone. This also eliminates the need for cumbersome chest-attachable AF monitors, bulky biofeedback devices, and separate vascular Doppler detectors. Monitoring various CVD indicators, which is often complicated, time-consuming, and costly, can instead be simplified to the easy placement of fingers on the OLED display, allowing users to perform assessments whenever and wherever needed.

Fig. 1: The concept of the sensor OLED-based multifunctional mobile cardiovascular health monitor.
figure 1

a The concept of the MCHM. b Technological impact of the Sensor OLED: Complicated cardiovascular health screening, blood pressure measurement for both arms, arrhythmia sensing, biofeedback therapy with multiple standalone devices and vascular Doppler detector are merged into a single mobile device which allows truly ubiquitous and convenient measurement (simply place a finger on the display).

Full size image

Implementation of the sensor OLED display

PPG sensing involves detecting volumetric changes in blood vessels resulting from regular heartbeats by measuring the intensity of scattered light that is absorbed. To enable this, the Sensor OLED was developed by integrating a massive array of OPDs with a resolution of 1024 × 1768 pixels (262 pixels per inch (ppi)) into a state-of-the-art flexible OLED display with a size of 7.5 inches and a resolution of 2160 × 1780 pixels (374 ppi), all while maintaining the same display performance. Details regarding the implementation of the Sensor OLED can be found in our previous reports4,5 and in Supplementary Figs. 5–12, including wavelength selection, implementation details with technical challenges, and enabling method for multiple objects sensing. Figure 2a illustrates the system block diagram, which includes the Sensor OLED display, integrated circuits (ICs), and software for operation, encompassing signal processing, algorithms, and the graphical user interface (GUI). Once PPG sensing begins, the OLED emits light, which is then scattered and reflected from the finger and detected by the OPDs. The readout operation is carried out by the readout IC (ROIC), followed by signal processing and algorithm execution to estimate the relevant indicators.

Fig. 2: Implemented sensor OLED display.
figure 2

a The operation principle of the Sensor OLED-based PPG sensing. b The system block diagram of the Sensor OLED, Multi-point sensing (MPS. c The operation procedure. d The operation timing diagram.

Full size image

In this section, we focus on introducing the implementation perspective of the MPS-capable Sensor OLED and the proposed methods for enhancing signal quality (SNR).

Figure 2b illustrates the detailed system block diagram of the Sensor OLED. As shown, the Sensor OLED consists of a panel that integrates the display and optical sensor in the same horizontal plane, a touchscreen panel (TSP) implemented on top, and the necessary electronics for display (driver IC (D-IC), and timing controller (T-CON)), optical sensing (ROIC), and the TSP (touch IC). In the panel, the display (D) and organic photodiode (O) pixels are arranged regularly and operate using shared identical control signals, which are internally generated by the display’s embedded scan driver (the gate control signals are labeled Gate1 to [N] in Fig. 2c). This design enables concurrent and synchronized display and sensing operations with a refresh rate of 120 Hz. The detailed operation procedures are illustrated in Fig. 2c. When the user places their fingers on the Sensor OLED to initiate sensing, the TSP and touch IC detect the touch positions. This coordinate information is then transferred to the application processor (AP) from the touch IC. The AP sends instructions to activate the green display pixels within a doughnut-shaped emitting pattern (approximately 9 mm in diameter for optimal SNR, as depicted in Fig. 2a, c5) at the corresponding sensing positions through the display electronics (T-CON and driver IC). Simultaneously, optical sensing is executed by reading the current generated from the OPDs located within the yellow inner circle area shown in Fig. 2a, c. This setup allows for fully time-synchronized multi-point PPG signals to be obtained for both multi-site and bilateral PPG and image sensing.

Figure 2d presents the operation timing diagram of the Sensor OLED. Once the touch enable signal is activated, it triggers the OLED and OPD operations by detecting and sending the touch coordinate information. A single frame of the display image for sensing (the emitting pattern) and the optical sensing data from multiple positions are displayed and sensed concurrently every 8.3 ms (refresh rate of 120 Hz, as shown in Fig. 2d). The sensed OPD signal is then quantized by the analog-to-digital converter (ADC) in the ROIC and transferred to the AP for pre-processing. This data is subsequently input into the biomarkers estimation algorithms, which will be detailed later.

Acquiring a high SNR signal is essential for ensuring high accuracy in both PPG and high-resolution imaging applications. Based on the simulation study conducted to examine the correlation between PPG SNR and BP estimation performance, found that the accuracy and reliability improved as the SNR increased (Supplementary Fig. 13). To address this, we proposed a new optical system designed to enhance signal quality in both areas, especially since the PPG signal was previously degraded to maintain image quality5. We developed a multi-layered Black Matrix (MLB) based optical system, achieving signal improvements of 20% in PPG and 15% in imaging applications. To further reduce noise, we optimized the Separator structure5, effectively minimizing leakage current through the shared common layers from the OLED to the OPD’s anode during display and sensor integration. This optimization resulted in a reduction of lateral leakage current, which contributes to noise, by increasing the width of the Separator. Details of these enhancements are provided in “Methods”.

As shown in Supplementary Table 1, the implemented Sensor OLED demonstrates superior performance in terms of size, user interactivity, resolution, refresh rate, and support functions.

MPS (1) Cardiovascular health monitor

Cardiovascular diseases are the leading causes of death worldwide. Among these, peripheral arterial disease (PAD) is particularly prevalent and associated with high mortality. PAD is characterized by arterial stenosis and vascular complications, which increase the risk of other CVDs, such as coronary artery disease (CAD) and cerebrovascular accidents (CVA). It occurs due to narrowed or blocked arteries from plaque buildup, potentially leading to serious complications like heart attacks and strokes. Given its high mortality rate and often asymptomatic nature, ubiquitous monitoring is essential.

Current diagnostic tools, such as vascular screening devices (e.g., the ankle-brachial index (ABI) and imaging devices (ultrasound, magnetic resonance angiography (MRA), and computed tomographic (CT) angiography), are time-consuming, expensive, and complex medical tests that can only be conducted in hospitals.

PPG pulse signals carry valuable physiological information related to heart function, circulation, respiration, the autonomic nervous system, and various diseases, making them useful for evaluating related risk factors. Additionally, simultaneous multi-site PPG measurements can provide more valuable insights. Previous studies have demonstrated significant differences in timing, waveform characteristics, and correlation coefficients between bilateral PPG measurements in patients with cardiovascular diseases, vascular diseases, hypertension, and diabetes11,12,13,14,15.

Figure 3a illustrates the principle of bilateral PPG sensing, showing how differences in PPG pulses can be observed based on the condition of the arteries. By obtaining and analyzing signals from the left and right sites using the MPS-capable Sensor OLED, we anticipate the significant potential for accurately screening CVDs, which conventional single-site PPG sensors cannot support. Figure 3b illustrates the operational principle of MPS sensing. When the user places their fingers on the Sensor OLED, the touch IC detects the touch positions. PPG sensing (OLED emitting and OPD sensing) is then executed to obtain fully time-synchronized bilateral PPG signals from the corresponding positions. Subsequent signal analysis is performed to screen for CVDs.

Fig. 3: Cardiovascular health monitor.
figure 3

a The concept of the bilateral PPGs depending on the different artery condition. b The concept of CVD screening by using Multi-point PPG sensing. c Different PPG pulse indexes used for CVD screening test. d Validation environment. e CVD screening validation results: AUSPRatio versus Rise time difference.

Full size image

Among the features of the PPG pulse waveform, we focused on those related to the rising edge of the pulse from the foot to the systolic peak, including the area under the systolic peak ratio (AUSPRatio), rising time (RT) difference, and correlation coefficient, as shown in Fig. 3c. These features are influenced by the pressure wave generated by the left ventricle as it travels to the extremities, providing direct insights into artery conditions, including vascular compliance and the degree of stenosis15,16. In contrast, features related to the diastolic peak, such as the reflection index (RI) and stiffness index (SI), were not considered, as they are significantly affected by age and thus may not be universally applicable across all age groups17.

To assess feasibility, we conducted a pilot study involving 88 subjects at Chungnam National University Sejong Hospital, following Institutional Review Board (IRB) approved procedures. The study included 47 subjects with CVDs such as PAD, CAD, and CVA, along with 41 healthy subjects. Detailed demographic information of the participants is provided in Supplementary Table 2. To confirm CVDs, the ankle-brachial index (ABI) was measured using a commercial vascular screening monitor (VP-1000plus, Omron, Japan). Following this, measurements with the Sensor OLED were conducted on the left and right index fingers, as illustrated in Fig. 3d.

The AUSPRatio18,19 is defined as

$${A}{US}{P}_{Ratio}=\frac{{AUS}{P}_{R}}{{AUS}{P}_{L}}=\frac{{\int }_{{t}_{0}}^{{t}_{p}}{{PP}{G}_{R},}_{F2P}(t)}{{\int }_{{t}_{0}}^{{t}_{p}}{{PP}{G}_{L},}_{F2P}(t)}$$
(1)
$${AUS}{P}_{{R}{atio}}=\left\{\begin{array}{c} > 1,{AUS}{P}_{R} \, > \, {AUS}{P}_{L}\\ \approx \!1,{AUS}{P}_{R}\approx {AUS}{P}_{L}\\ < 1,{AUS}{P}_{R} \, < \, {AUS}{P}_{L}\end{array}\right.$$
(2)

where AUSPR and AUSPL are the areas under the systolic peak of the right and left finger PPG, respectively; PPGR,F2P(t) and PPGL,F2P(t) are the PPG signals from the right and left pulse foot (F) to systolic peak (P), respectively and [t0, tp] is the time interval. AUSPratio >1 or <1 represents bilateral differences in CVD screening. The RT difference is also defined as

$$\Delta {\rm{RT}}={{RT}}_{R}-{{RT}}_{L}$$
(3)

where RTR and RTL are the rising time (foot (F) to systolic peak(P)) of right and left PPG. The correlation coefficient is a linear correlation between two pulse’s amplitude. We compared the detection accuracy using AUSPratio, RT difference, and correlation coefficient. The CVD screening thresholds were set as follows: an AUSPratio value less than 0.95 or greater than 1.05 indicated a positive screen; an RT difference greater than 33 ms was also considered positive; and a correlation coefficient less than 0.97 indicated CVD. The AUSPratio threshold was determined based on the robust linear model (RLM) evaluation, which confirmed a significant difference between the CVD and normal groups. This finding was further validated by a 95% confidence interval, ensuring the reliability of the result (Supplementary Table 3). The results are summarized in Table 1. As shown, the AUSPratio demonstrated the best CVD screening capability, with a precision of 94.44%, recall of 82.93%, accuracy of 90.08%, and F1 score of 88.31%, indicating high accuracy. In contrast, the detection accuracy for the RT difference and correlation coefficient was lower. Figure 3e illustrates the correlation between AUSPratio and RT difference. It shows that as the RT difference increases, the AUSPratio value can either increase or decrease. Notably, even with the same RT difference (less than 40 ms), differences in AUSPratio were observed between the normal and patient groups. This leads us to conclude that the area-related index (AUSPratio) is more suitable and effective for CVD screening compared to timing-related indices, consistent with findings in ref. 20.

Table 1 Performance comparison with the different PPG pulse indexes
Full size table

In conclusion, the smart device utilizing the MPS-capable Sensor OLED has proven effective for ubiquitous CVD screening. In terms of usability, the Sensor OLED offers significant convenience, requiring only 15 s for measurements (as shown in Supplementary Table 4), with no restrictions related to place, time, or cost. In contrast, conventional CVD monitors typically require 5 to 10 min and are subject to various limitations, including the need for hospital visits, time constraints, and associated costs (see Table 2).

Table 2 Usability comparison with the conventional devices
Full size table

MPS (2) Blood pressure monitor

Hypertension, often referred to as a silent killer, is prevalent worldwide and is a significant risk factor for CVDs. Accurate blood pressure measurement is essential for diagnosis and management. Since hypertension can be modified through lifestyle changes and medication, ubiquitous BP monitoring is crucial for individuals with related symptoms. However, conventional cuff-based BP monitors, despite their high accuracy and routine use in clinical practice, face significant limitations in everyday usability. These include bulkiness, discomfort, the need for user proficiency, and limited accessibility, particularly in low-resource settings. To address these issues, several technologies have recently been developed to estimate BP using sensors, signal processing, and algorithms embedded in mobile and wearable devices (see Supplementary Table 5)21,22,23. Cuffless BP monitoring technologies promise to measure BP comfortably with minimal user intervention by eliminating the need for a cuff, thereby facilitating ubiquitous self-monitoring. This approach can enhance early awareness and treatment of hypertension, as well as improve adherence to treatment and long-term BP control. Recently, cuffless devices have increasingly emerged in the market. However, significant accuracy-related challenges must be addressed to enable widespread adoption in clinical practice24,25.

The accuracy issues are illustrated in Fig. 4a–c. Figure 4a shows a scatterplot comparing cuffless BP measurements with reference cuff BP, demonstrating a strong correlation across all subjects’ BP values. However, the correlation within individuals is close to zero. This discrepancy arises because current accuracy tests are conducted immediately after calibration in static conditions, resulting in values that are consistently close to the calibrated BP—an approach that is insufficient for validating true accuracy.

Fig. 4: Blood pressure monitor.
figure 4

Three accuracy issues a Intra-person BP change. b Track BP change. c Long-term stability (One person conducted measurements three times a day for 29 days, the dots represent the mean value and the bars represent the 95% confidence interval). d The proposed BP sensing methods. e The proposed model network, f Bland-Altman Plots (SBP and DBP) for single-point sensing.

Full size image

To accurately assess performance, additional testing during BP fluctuations within each individual is necessary, as indicated in Fig. 4b. However, interventions to induce changes in BP—whether increases or decreases—can pose risks to subjects, and no specific methodologies have been established yet. Furthermore, cuffless devices require periodic user calibration every few weeks (varying by manufacturer), making stability tests immediately before the recommended recalibration is essential to ensure prolonged accuracy throughout the calibration period, as illustrated in Fig. 4c.

To address these issues, we employed two different approaches, as shown in Fig. 4d. The first approach involves developing a differentiated algorithm for single-point BP sensing, using a single calibration and a single-point PPG signal. The second approach utilizes both left and right PPG signals as inputs with a single calibration PPG, aiming to enhance estimation accuracy by effectively doubling the input information and maximizing the MPS capability of the sensor OLED.

For the single-point BP sensing, we focused on three different aspects to improve accuracy and validate performance.

Firstly, for the dataset, we employed PulseDB26 for training and validation in a subject-independent manner, ensuring that no individual data from the training set appeared in the validation set. This approach is crucial for improving accuracy, as it reflects real-world clinical scenarios where patient-specific data may not be readily available. For testing, we used a dataset obtained from a pilot study at Samsung Medical Center, which fully complies with regulatory standards27 for medical approvals. Additionally, we conducted a four-week longitudinal study involving over 10% of the total subjects to validate the algorithm’s capability to track BP changes and assess long-term stability.

Secondly, we developed a differentiated pre-processing and machine learning (ML) model. Supplementary Fig. 14 illustrates the pre-processing flow, which incorporates several critical stages to refine and prepare the dataset for optimal model training and validation. The initial steps involve removing abnormal data, including mislabelled samples, low-SNR signals, and segments with abrupt BP changes. This is followed by filtering to extract physiological signals and applying ensemble averaging to obtain robust representative signals from the repetitive PPG pulse train, effectively reducing noise. Finally, the segments are divided into training and validation sets in a subject-independent manner, minimizing the risk of model overfitting and enhancing generalizability to new, unseen data. We developed a new ML model called SOLED2DeepBP, which aims to account for the temporal dynamics and salient features of PPG waveforms for accurate BP estimation. We applied a Siamese network for the two PPG signals (ensemble target PPG and calibration PPG), utilizing a one-dimensional convolutional neural network (1D-CNN) based on the ResNet framework, along with a long short-term memory (LSTM) network and an attention layer for further enhancement, as shown in Fig. 4e. Features extracted from the calibration systolic and diastolic BP using a multilayer perceptron (MLP) are concatenated with the features derived from the PPG signals via the Siamese network. These combined features then pass through a fully connected layer (FCL), resulting in the estimation of the difference in BP between the calibration and target measurements, whereas conventional algorithms estimate absolute BP values. This integrated approach—combining the strengths of convolutional feature extraction, temporal sequence modeling, and focused feature selection—maximizes the accuracy of BP estimation.

Lastly, a pilot study was conducted in strict adherence to standard protocol27, including an additional four-week follow-up test, as shown in Supplementary Fig. 15. This was done to validate the algorithm’s accuracy, track BP changes, and assess long-term stability, ensuring that it reflects real-world clinical scenarios.

As a result, the algorithm achieved a mean error (ME) and standard deviation error (STDE) for systolic BP (SBP) of 0.18/6.8 mmHg and for Diastolic BP (DBP) of −0.02/4.8 mmHg across the entire dataset (as shown in Fig. 4f, which illustrates the significant BP variations in the test dataset, with an STD of SBP: 18.2 mmHg and DBP: 12.7 mmHg, detailed in Table 3 and Supplementary Table 6). In the follow-up set, the ME and STDE for SBP were −0.47/6.9 mmHg and for DBP −0.47/6.31  mmHg, as summarized in Table 3 (Supplementary Fig. 16).

Table 3 Performance comparison with the current state-of-the-art BP estimation algorithms (single-point sensing)
Full size table

These results met the Association for the Advancement of Medical Instrumentation (AAMI) standards (ME and STDE of ≤ ±5/8 mmHg), confirming that the algorithm complies with regulatory requirements for medical devices. In contrast, the current state-of-the-art algorithm28 did not meet these specifications, particularly with the wide BP distribution, as shown in Table 3. To further verify the algorithm’s capability to track intra-subject BP changes, we calculated the correlation coefficient values for each individual subject’s BP. As shown in Supplementary Table 7, our algorithm achieved correlation coefficients of 0.7 for systolic BP and 0.56 for diastolic BP. In contrast, the conventional algorithm exhibited much lower coefficients of 0.04 for systolic BP and 0.01 for diastolic BP, particularly in the subject with the largest BP variation (SBP STD: ±21.26 mmHg, DBP STD: ±16.54 mmHg).

For the second approach, dual-point BP1 sensing (Fig. 4d), we estimated BP values from bilateral PPG signals using a single calibration PPG signal as input for the designed BP algorithm. The final BP value was determined by averaging the estimates from the left and right measurements. To validate the effectiveness of this approach, we conducted a pilot study with 57 participants. Table 4 summarizes the performance comparison between single-point and dual-point BP sensing. While the dual-point approach (Fig. 4d and Supplementary Fig. 17) significantly reduced the sensing failure rate from 8.82% to 2.75%, it did not substantially change the overall performance of BP estimation. The reduced failure rate is attributed to the doubled information provided by the two fingers, which ensures reliable estimation even if one signal is impaired due to artifacts or low SNR. However, in cases where vascular diseases are present in one or both fingers, the average BP value could potentially be unreliable. In such scenarios, analyzing the differences between bilateral PPG signals might provide valuable diagnostic insights, particularly for detecting asymmetrical vascular conditions. This highlights the need for future studies to explore the diagnostic potential of bilateral signal comparison rather than relying solely on averaging.

Table 4 Performance comparison: single point vs. dual-point BP1 sensing
Full size table

In conclusion, the differentiated BP algorithm was developed and thoroughly validated, demonstrating performance comparable to medical devices while addressing the previously mentioned accuracy challenges through the pilot study that reflects real user environments. The unique ML approaches and their effectiveness over conventional algorithms are summarized in Supplementary Table 8. Additionally, BP readings obtained from both fingers, utilizing the distinctive MPS capability of the Sensor OLED, enhanced both accuracy and usability. We believe that developing a more advanced end-to-end ML algorithm specifically designed for bilateral BP estimation—utilizing both bilateral PPGs and calibration PPGs as inputs (as illustrated in Fig. 4d, dotted box)—could further enhance performance by leveraging the interrelationships between the PPG signals. It will be studied further in the follow-up research.

MPS (3) Blood vessel imaging and blood flow dynamics

The Sensor OLED can also capture high-resolution images, such as fingerprints, by modifying its OLED emitting, OPD sensing, and signal processing methods. This is achieved by detecting differences in reflected light, distinguishing between diffuse and specular reflection from the emitted OLED light, as illustrated in Fig. 5a. Further details about the multifunctional capabilities of the Sensor OLED can be found in ref. 5. As shown in Fig. 5b, consecutive high-resolution image captures from the fingers enable identification of blood vessels. Additionally, by analyzing the gray level changes in these images in relation to heart palpitations, blood flow dynamics could be detected through the generated pulsatile (AC) signal map. Consequently, we believe the Sensor OLED embedded smartphone could possibly complement traditional vascular Doppler detectors, typically only available in hospital settings, making it useful for diagnosing finger blood vessel conditions and assessing circulation quality.

Fig. 5: Blood vessel imaging and blood flow dynamics.
figure 5

a The system block diagram for high-resolution dynamic imaging sensing (HDIS). b Multi-finger blood vessel imaging and blood flow dynamics sensing results (principle and measurement results). c HDIS-based Biosensing : a spatial sensing in the multiple positions within the sensing area and spatial-temporal sensing are possible due to the abundant information captured.

Full size image

Moreover, new biosensing methods have been developed based on the generated AC signal maps. As can be seen in Fig. 5c, the conventional PPG signal is captured from a single dedicated point due to the limited channel, so the single-channel PPG pulse train is the only obtainable signal, which has certain limitations. However, the Sensor OLED can obtain abundant optical image data from the massively pixilated sensors, allowing for spatial sensing from multiple positions within the sensing area, as multiple PPG pulse trains can be obtained through image processing. Additionally, analyzing the time series of AC signal maps could enable spatial-temporal sensing. We believe it has various potential applications, such as in systemic sclerosis6, peripheral arteriolar lesions7, and Buerger’s disease8, where blood flow dynamics and blood vessel image sensing are required. We also believe that multi-point HDIS sensing could provide additional useful information. Furthermore, it may be applicable to authentication and anti-spoofing applications (Supplementary Figs. 1 and 2). In this paper, we remain open to exploring the application potential of these sensing methods, and we plan to conduct feasibility studies to identify potential applications and validate their effectiveness in the near future.

UIS

Due to the benefits of display and sensor integration, real-time interactive sensing is supported while the signal capture is in progress. Users can check the signal quality in real-time while observing the GUI on display and take immediate corrective actions, such as modifying finger contact conditions, including finger position and pressure. With our internal testing, we achieved a 0% PPG sensing failure rate, down from 14.5%, after adopting the interactive sensing scheme that ensured secured PPG signal quality (SNR ≥ 20 dB), as detailed in Supplementary Fig. 3. Moreover, the finger produces a clearer and more reliable PPG signal with highly informative waveform features (including mean amplitude, pulse peak time, dicrotic notch time, and reflection index) than the wrist. This is due to its greater sensitivity to volumetric fluctuations in blood, attributed to the rich arterial supply and thinner tissue thickness29. To fully leverage these advantages, arrhythmia, stress, heart rate, and respiratory rate sensing were implemented, and their performance was validated (Detailed in Methods). In particular, stress sensing with the Sensor OLED could enable a truly single smartphone-based biofeedback device (BFD), replacing the currently prevalent bulky, multiple-device-based BFDs (Supplementary Fig. 4c).

Discussion

We presented the Sensor OLED-based, single mobile device-type multifunctional CVD health monitor. The Supplementary Movie 1 demonstrates the operation. Due to the distinctive advantages of high-resolution pixilated integration of the OPD sensor within the full display area, the multi-point PPG sensing achieved high accuracy (90%) in CVD screening and improved accuracy in BP monitoring (reducing the mean and standard deviation errors), as well as enhanced usability (reducing the failure rate from 8.82% to 2.75%). The effectiveness of multi-point high-resolution dynamic image sensing, along with the developed new sensing methods (spatial and spatial-temporal sensing), will be explored in the near future. Additionally, the user-interactive sensing resulted in secure high PPG quality, with a remarkably improved sensing failure rate. In conclusion, these advances enabled new functions, as described so far, with medical device-level performance based on the validation results obtained from pilot studies. As summarized in Table 5, the benefits of the Sensor OLED are clearly demonstrated in terms of its supported sensing capabilities, functions, and ubiquitous sensing ability. Integrating it into a smartphone—an indispensable device in our daily lives—effectively eliminates the burden of carrying standalone devices or separate sensors. Moreover, it offers easy usability, with a short sensing time of just 15 s and no restrictions when using a single smartphone, as summarized in Table 2, while supporting multiple functions. As a result, we believe the Sensor OLED could be a new enabler in the evolution of smart devices toward mobile health monitoring and digital therapeutics, potentially revolutionizing diagnostics and treatment. Consequently, it could notably reduce the healthcare burden on society.

Table 5 Performance comparison with the current state-of-the-art PPG sensors
Full size table

Limitations and future directions

The following limitations and future directions will be addressed to improve the technology’s completeness:

  • Pilot studies conducted for Korean, will expand into formal clinical studies involving diverse racial groups to ensure reliable performance across different skin tones.

  • The CVD pilot study showed initial feasibility; the next steps include larger samples, severity analysis, refined algorithms, and clinical validation.

  • The single-point BP study followed the standard protocol, while the dual-point method was tested in a pilot study. Future work will improve the algorithm using PPG signals from both fingers, and a clinical study will validate it.

  • No specific applications and validation studies have been conducted in HDIS yet, but it will be followed.

  • UIS currently uses PPG SNR to assess signal quality but will add PPG morphology for better quality in the future.

Methods

High SNR sensor OLED implementation

The proposed SNR improvement methods aim to optimize sensing performance in both PPG and high-resolution imaging applications, such as fingerprint and HDIS sensing. These methods include (1) A new optical system to enhance the signal in both applications and (2) An improved Separator structure to reduce lateral leakage current and effectively minimize noise. Supplementary Table 9 highlights the key differences between fingerprint and PPG sensing, such as sensing objects, methods, key performance measures, and frame rates, which often conflict. Fingerprint sensing detects the difference in reflected light from the fingerprint ridge (diffuse reflection) and valley (specular reflection) under OLED light. PPG sensing, however, detects volumetric changes in blood vessels due to heart palpitations by analyzing the intensity of scattered light. Because of these differences, selecting the optimal optical system design is critical. The BM hole size (p) plays a key role, with fingerprint sensing requiring precise differentiation between ridges and valleys (10 × 10 μm2) to avoid image blurring or SNR degradation. However, using this design in PPG sensing reduces light intensity, which affects signal quality. Previously, we optimized the optical system for fingerprint sensing across the entire display area, compensating for PPG signal degradation using multiple lines integration (MLI) in the ROIC (Supplementary Fig. 6). However, this could not overcome signal reduction due to the small BM hole width (p). A significant challenge is the sensing range threshold (~600 μm) for high-resolution imaging applications, where values larger or smaller than this threshold cause image blurring or SNR degradation. S can be represented by the simple formula as30,

$$S=p+\frac{L}{l}\left(p+s\right)$$
(4)

where, p is the BM hole width, s is an OPD pixel size, \(l\) is a thickness from the BM hole to OPD pixel layer and L is the thickness between the BM hole and the cover glass. The new optical system controls the layer thickness (l) to maintain the sensing range within the threshold ( ~ 600 μm) while increasing the BM hole width (p) to enhance the signal. (Supplementary Fig. 7) By adjusting design parameters in the multi-layered black matrix (MLB) system (\(l\) → 2.5\(l\)), we achieved a sensing range below the threshold with a larger p (up to 15μm), improving PPG signal quality by 20% and fingerprint/HDIS signal quality by 15% compared to the conventional single-layered black matrix (SLB). Detailed results are shown in Supplementary Fig. 7c, d. Additionally, minimizing lateral leakage current from the OLED to the OPD’s anode is crucial for improving SNR. To address this, we introduced a Separator with a reversed taper shape to disconnect the common deposition layers between the two devices, effectively reducing leakage current (Supplementary Fig. 9). Increasing the Separator width further reduces leakage, making it especially beneficial for high-resolution OLED sensors, where the OLED and OPD are closer (Supplementary Fig. 7e).

System development

The implemented system was comprised of the Sensor OLED, Field Programmable Gate Array (FPGA) board, and PC. The FPGA board was used for controlling and communicating data between the Sensor OLED and PC. A Universal Serial Bus (USB) connection was employed between the FPGA board and the PC for data communication. The software was developed in C# and Python. A custom software made by C# was used to control the FPGA and the Sensor OLED. Python programmed algorithm was employed for the multiple CVD indicators sensing.

CVD health monitor validation

The pilot study was conducted and was approved by the Institutional Review Board (IRB) at Chungnam National University Sejong Hospital (CNUSH) (IRB File No: 2024-03-002-002). The experimental procedure was explained, and written informed consent was obtained from all participants (88 subjects). The participants were divided into two groups based on whether they had a history of CVD, specifically CAD, CVA, or PAD. A total of 47 subjects with cardiovascular diseases were recruited, while 41 healthy individuals were included as the control group. The ankle-brachial index (ABI) measurement was performed using the Omron VP-1000 Plus (Omron, Japan) over a duration of 30 min to determine whether the subjects had CVD. After this test, the participants rested for 5 min before sitting in a chair. Bilateral PPG signal measurement with the Sensor OLED was then executed by placing the left and right index fingers on the display for 30 s. After the first measurement, wait for 2 min before measuring the signal again for 30 s. The participants sat on a chair with their backs straight, placed their fingers on the display, minimized movement, stayed relaxed, and breathed naturally during the measurement.

BP algorithm—training and validation sets

Derived from the open dataset (PulseDB), we meticulously curated our dataset by excluding cases with low SNR or incorrect labels, resulting in a refined collection of 938 subjects (106,609 samples) for training purposes. Additionally, 403 subjects (45,159 samples) were selected to validate the model (Supplementary Table 6).

BP algorithm—test set (clinical data, single-point BP sensing)

To evaluate the model’s capability to meet the regulatory requirements for BPM, the pilot study was performed which was approved by the IRB at Samsung Medical Center (SMC) (IRB File no: 2023-07-129). The experimental procedure was explained, and written informed consent was obtained from all participants (120 subjects). It was conducted in accordance with the ISO 81060-2 protocol. The data collection, referred to as the whole set, aimed to determine the model’s applicability in real-world conditions. Further, to assess the model’s robustness over time, a Follow-up set was obtained by collecting additional data from 15 participants (over 10% of the total subjects) across a 4-week period. This strategic approach ensures a thorough validation of the model’s efficacy and stability.

PPG signal pre-processing for the BP algorithm

It incorporates several critical stages to refine and prepare the dataset for optimal model training and validation. (i) Removal of abnormally labeled samples: Initially, we eliminate samples with labels indicating biometric or measurement values outside plausible ranges. This includes samples where the weight is below 10 kg or above 100 kg, height is under 100 cm or over 200 cm, SBP falls below 70 mmHg or exceeds 200 mmHg, DBP is lower than 30 mmHg or higher than 150 mmHg, and instances where BP or PPG data are missing. (ii) Exclusion of low-SNR segments: we conducted Welch’s method for power spectrum analysis to calculate the SNR. The data below a predetermined threshold are excluded. (iii) Elimination of segments with abrupt BP changes: Segments exhibiting abnormal BP changes within a short time period were removed. This step is crucial to discard the data that may represent measurement errors or physiological anomalies not relevant to the typical BP variation patterns. (iv) Filtering: a fourth-order Butterworth IIR band-pass filter with a passband of 0.8 to 6 Hz was applied. It aimed to remove the frequency components that are not relevant to the physiological signal, thereby enhancing the signal quality for subsequent analysis (the same filtering method was applied in CVD monitoring application) (v) Extraction of representative signals with the ensemble averaging (EA): As depicted in Supplementary Fig. 14, aligning and averaging multiple signal epochs (periods of time for the repetitive signals), allow reinforcement of the common signal components, while the random noise due to the motion, ambient light, electronic circuits, etc. is removed. This technique is important for improving algorithm’s performance by extracting the consistent patterns of the blood volume pulses. (vi) Subject-independent dataset segmentation: The data segmentations were divided into the training and validation set in a subject-independent manner to minimize the risk of model overfitting and enhance its generalizability to new unseen data.

BP pilot study for validation—single-point BP sensing

It was carried out at Samsung Medical Center and targeted adults aged 19 to 70 years. The participant enrollment followed guidelines (a minimum of 85) specified by the AAMI/European Society of Hypertension/International Organization for Standardization (AAMI/ESH/ISO). Participants’ basic demographics, medical histories, comorbidities, and medication usage were evaluated through interviews conducted by the physicians and documented electronically. Prior to the BP measurement, participants sat comfortably for at least 5 min. During the measurement, participants were positioned with their back and arms supported, left arm at heart level, legs uncrossed, and feet flat on the floor. Interference during measurement was strictly prohibited. Firstly, test device calibration was performed immediately before the BP measurement. Test BP, measured by the Sensor OLED device, was compared with the reference BP, measured by the reference auscultatory cuff device (BPBIO210, InBody Co, Korea). Reference SBP and DBP were determined using a double-headed stethoscope for simultaneous auscultation, by the two trained nurses who were blinded to each other’s readings, with the first and fifth Korotkoff sounds recorded as SBP and DBP, respectively. The average BP values from both observers served as the reference for analysis. Test and reference BPs were measured concurrently, with the participants holding the test device in their right hand and positioning their index finger on the Sensor OLED to obtain the PPG signal. The PPG signal from the Sensor OLED was obtained for 15 sec. Participants were corrected if their position was inadequate for measurement. This study comprised two phases. The primary phase aimed to assess the accuracy of the test device compared to the reference device across the entire study population (N = 120). Conversely, the secondary phase sought to verify the stability of the test device over a four-week period following the primary phase. During the primary phase, each participant underwent three BP measurements after individualized calibration. The calibration involved putting the measured reference BP value into the test device after simultaneous measurement of the test and reference devices, repeated three consecutive times. Following calibration, participants rested for 5 min before three consecutive BP measurements were taken, with 5-min intervals between each measurement. This process yielded a total of 360 pairs of test-reference blood pressure sets for SBP and DBP, respectively, from the 120 participants. Additionally, BP was recorded weekly for four weeks in 15 initially enrolled participants to investigate the test device’s stability over time. Follow-up measurements were identical to the initial BP measurements, except that no calibration was performed. (Supplementary Fig. 15).

BP pilot study for validation—dual-point BP1 sensing

To validate the effectiveness of the proposed dual-point BP sensing, the pilot study was carried out. The experimental procedure was explained, and written informed consent was obtained from all participants. A total of 57 participants took part in the experiment for four weeks, visiting once a week to measure their blood pressure. The measurement has proceeded with the reference device (OMRON HEM-7120), and then PPG sensing from both fingers using the Sensor OLED for comparative evaluation was executed. This procedure was repeated 12 times (three times in a row with 30-min rest between the intervals) per visit (once a week) and continued for 4 weeks. The demographic information is tabulated in the Supplementary Table 10.

Atrial fibrillation (AF)

It results from irregular electrical signals in the SA node and increases with age. AF is a major stroke risk, contributing to 20–30% of cases31 (Supplementary Fig. 18a). Early detection is challenging due to its sporadic nature, with ECG monitors often inconclusive. PPG sensors, especially in wristbands, are increasingly used for AF detection, but issues like irregular heart rates and noise complicate this process. Interestingly, intermittently measured devices often yield cleaner signals, improving detection rates32. The Sensor OLED, with real-time PPG monitoring, provides cleaner signals, making it ideal for AF detection. By analyzing the autocorrelation of these signals using a deep learning model, we improved AF detection. Data pre-processing involved an IIR band-pass filter, normalization, and removal of low-SNR signals. A CNN model was used to differentiate normal rhythm from arrhythmia, trained on signals from wrist-based PPG devices and the 2015 IEEE Signal Processing Cup database33. Our algorithm, SOLED2DeepAF (Supplementary Fig. 18b), achieved 93% precision, 96% recall, 95% accuracy, and 95% F1 score (Supplementary Table 11), comparable to the DeepBeat model34. Unlike DeepBeat, which uses two complex models, ours achieves similar results with a simpler approach. The Sensor OLED could effectively replace traditional ECG-based AF monitors.

AF signal generation for validation

Since no clinical test for the AF algorithm validation was performed. So, the simulation test with the reproduced pulses utilizing the Sensor OLED was executed. The original PPG pulses (total of 50: 30 and 20 pulses for w/ and w/o AF respectively) selected from the PPG database(Supplementary Table 12) were inputted to the simulation system, which comprises of the Liquid Crystal Display (LCD) module and Micro-controller unit (MCU). Depending on the pulse input to the MCU unit, it controls the LCD module’s transmittance on top of the Sensor OLED so as to control the amount of reflected light to the OPD to generate PPG pulse. As a result, the replicated PPG pulses, which include unique noises (mainly the high-frequency noises) from the Sensor OLED were generated for the algorithm validation. (Supplementary Fig. 19).

Pulse rate variability (PRV)

It is a key metric for assessing autonomic nervous system balance and stress response. It helps evaluate mental health by analyzing heart rate fluctuations, with elevated stress typically reducing PRV (Supplementary Fig. 4a). Accurate PRV measurement requires high-quality PPG signals, and the Sensor OLED can facilitate precise assessments, aiding digital therapeutic development. PRV was computed using the pyHRV toolkit35 with R peaks and mean R-R intervals. Validation with 25 participants (Supplementary Table 13) showed a mean absolute error of 6.2 ms and a correlation coefficient of 0.99 (Supplementary Fig. 4b). The error is negligible given the Sensor OLED’s sampling rate of 120 Hz (approximately 8.3 ms per interval). Stress sensing can be integrated into a single biofeedback device (BFD) using a smartphone, enabling real-time, interactive biofeedback anywhere, leveraging the Sensor OLED’s PPG sensing capabilities (Supplementary Fig. 4c).

PRV and HR validation

To validate it, a pilot study was carried out. The experimental procedure was explained, and written informed consent was obtained from all participants. A total of 25 participants were recruited to assess PRV and HR (Supplementary Table 13). Each participant took three measurements using the medical device (uBioMacpa, BioSense Creative, Korea) and the Sensor OLED. The PPG signal from the left index finger was measured by the Sensor OLED, while the signal from the right index finger was measured by the medical device concurrently. Heart Rate (HR) showed a 0.99 correlation with the reference device (uBioMacpa). Bland-Altman plots are shown in Supplementary Fig. 22.

RR monitoring and validation

The RR monitoring is done by extracting the envelope signal with its frequency analysis (Supplementary Fig. 20). The same procedures for PRV and HR were repeated for validation with the reference device (RESP100 Biopac, USA) to verify the function. RR showed a 0.91 correlation with the reference device (Supplementary Fig. 21). Bland-Altman plot is shown in Supplementary Fig. 22.

Ethics

Every pilot studies have been carried out following protocols approved by the ethical commissions. Each participant gave informed written consent.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.