Introduction

Tools for clinical decision support (CDS) and remote patient monitoring (RPM) have the potential to facilitate population-level care management. Tools for CDS aid clinicians with interpretation, diagnosis, and treatment while RPM allows clinicians to monitor patient conditions remotely, both promising better patient care1,2. However, implementing technology-enabled care models and facilitating change management in health systems require overcoming technical, social, and organizational challenges3. Technical, infrastructural, psychological, and workload-related concerns are the top barriers to utilizing health technologies by healthcare professionals4. Though commercial diabetes RPM solutions exist, the majority of the 29.7 million patients with diagnosed diabetes in the US and the clinics that provide their care do not currently deploy technology-enabled whole-population care5. A recent independent evaluation of the clinical effectiveness and economic impact of eight commercial digital diabetes management tools (Glooko, DarioHealth, Omada, Perry Health, Teladoc (Livongo), Verily (Onduo), Vida, and Virta) found sufficient evidence to support limited broader adoption only of Virta6. While reports show that the majority of clinicians plan to use RPM tools, large-scale pragmatic RPM implementation has been limited7,8. The full potential for better glycemic control facilitated by continuous glucose monitors (CGM, small wearable devices that measure glucoses every 5–15 minutes) and insulin pumps has not been attained, due in part to clinicians lacking adequate time to translate the large volume of data from such devices into improved patient care9.

Timely Interventions for Diabetes Excellence (TIDE) is a CDS and RPM tool that facilitates personalized care for youth with type 1 diabetes (T1D) as part of the Teamwork, Targets, Technology, and Tight Control (4T) Study10,11,12. TIDE automates CGM data analysis and facilitates population health management by identifying patients at higher risk of deteriorating glucose management11. In a pilot and a follow up study, as part of the broader 4T pragmatic clinical trial (NCT03968055, NCT04336969), use of TIDE contributed to progressively lowering mean patient HbA1c, increasing patient time in range, and reducing mean time spent by providers reviewing patient data12,13,14. The care model has the potential to be financially sustainable at Center for Medicare and Medicaid reimbursement rates15. In a recent study, the 4T Study 1 cohort (36.8% Hispanic and 35.3% publicly insured) had a mean HbA1c of 6.58%; 64% with HbA1c < 7% and mean time-in-range (70–180 mg/dl) of 68% at 1-year post-diagnosis10,11,12. TIDE demonstrates that algorithms can translate voluminous data generated by CGM into actionable insights for providers and help direct limited clinical resources towards patients in the population most in need, i.e., the right care for the right patient at the right time10,11,16,17. A recent review of TIDE and the 4T program, with an emphasis on its potential to increase access to technology-enabled type 1 diabetes care, concluded “Nearly three decades after the pivotal Diabetes Control and Complications Trial—which firmly established the benefits of intensive management on diabetes outcomes—the prospect of mimicking the close monitoring and positive outcomes of that randomized, controlled trial may be feasible in clinical practice”18.

Numerous diabetes clinics have expressed interest and attempted to adopt the TIDE platform, including centers in the United States, Australia, and Canada. The research team has supported these efforts with a variety of nationally and locally funded efforts19,20,21. TIDE is available as a free, open source tool on GitHub to any clinic interested in replicating this part of the 4T care model14,16,22. An interactive version of TIDE with synthetic data is available on Amazon Web Service (AWS) for testing by potential users and an interactive web tool for designing an operational model to deploy TIDE is available online11,23,24,25. However, implementing a TIDE-based care model, or any RPM-based care model, involves significant organizational effort and the development of operational and technical pipelines to obtain patient consent and import and process patient data. Many clinics do not have access to such resources, and those that do or can afford them often face other hurdles3,4. Efforts to scale TIDE continue to face delays.

We sought to create a turnkey solution that could make personalized care at population-scale effective and efficient as well as readily adoptable across health systems. The Stanford 4T team partnered with Tidepool, a diabetes technology non-profit on a mission to “make diabetes data more accessible, actionable, and meaningful for people with diabetes, their care teams, and researchers.” Tidepool's products include Tidepool Loop, the first FDA-approved fully interoperable automated insulin dosing application and Tidepool+ , a class I FDA-approved diabetes data management platform26,27,28. Tidepool+ is available to US-based health systems, and aggregates and visualizes data from over 85 diabetes medical devices including CGMs, insulin pumps, and blood glucose meters as well as being one of the first platforms to offer access to continuous, cloud-to-cloud data from both Dexcom and FreeStyle Libre sensors29,30,31. A full, regularly updated list of compatible devices is available online31. Currently, Tidepool+ supports diabetes data management for over 1000 clinics and more than 600,000 people living with diabetes in the US.

Common barriers to the adoption and sustained use of digital technology have been documented3,4,32. The technical, clinical, financial, and operational innovations that allowed TIDE to overcome these barriers at a single institution have been reported10,11,12,13,14,15,16. This work presents how we addressed the barriers specific to scaling digital health technology from one site to many to create Tidepool-TIDE, a nationally-available turnkey tool.

Results

Tidepool-TIDE retains the functionality that helped improve outcomes at a single site and introduces functionality to address the barriers specific to scaling digital health technology from one site to many. These barriers include: psychological barriers such as difficulty understanding how the technology fits into a new care model and uncertainty about its efficacy in a new population; lack of or differences in the technical infrastructure and expertise necessary to deploy and support the technology; and concerns about the workload required to learn and use the technology (Table 1)3,4,32.

Table 1 Common barriers to the adoption of digital technology developed at a different institution and how they are addressed
Full size table

Difficulty understanding how the technology fits into the local care model

We distilled site-agnostic, fundamental top-of-license components of whole-population CGM-enabled care through extensive shadowing and structured interviews of Certified Diabetes Care and Education Specialists (CDCES), the primary intended users of Tidepool-TIDE. In the triage phase, they determine which patients in the population should be prioritized for support. In the assessment phase, they determine the status of the patient’s T1D management using detailed individual-level data. In the investigation phase, they use CGM data along with their knowledge of the patient to identify the challenges driving suboptimal glycemia for the patient. In the intervention phase, they determine how to support the patient in improving glucose management and communicate with the patient through a secure message or phone call (Fig. 1). The instructional materials generated to support Tidepool-TIDE identify how its use enables each of these phases of care.

Fig. 1: CDCES patient review flow.
figure 1

Fundamental components of a whole-population CDCES-led care model include triage, assessment, investigation, and intervention.

Full size image

During these interviews we determined that Tidepool-TIDE should retain the transparent, interpretable metrics used in TIDE and add the widely used Glucose Management Indicator (GMI) metric. The metrics retained from TIDE are Advanced Technologies and Therapeutics for Diabetes (ATTD) consensus targets, a personalized metric comparing time in range across consecutive weeks, and CGM wear time (Table 2, Fig. 2)33. The consensus targets used are: total time severely hypoglycemic (glucose < 54 mg/dL) < 1%; total time hypoglycemic (glucose < 70 mg/dL) < 4%; total time in range >70%. The personalized metric is defined as a drop in time in range > 15% points. Low CGM wear time is defined as CGM wear time <70%. Orienting care around transparent metrics removes uncertainty about how patients are ranked or the role of the technology. Patients achieving these metrics during TIDE review signify to CDCES that additional review of CGM data may be better directed towards other patients.

Table 2 TIDE and Tidepool-TIDE functionality to support clinical, administrative, and process improvement workflows
Full size table
Fig. 2: Tidepool-TIDE dashboard.
figure 2

Tidepool-TIDE dashboard with synthetic data. Risk stratification is by ATTD consensus glucose metrics and a personalized time-in-range metric. Of patients with sufficient CGM wear time, patients with excess severe hypoglycemic are assigned highest priority, followed by patients with clinically significant hypoglycemia, patients with a large drop in time-in-range compared to the prior week, patients with overall time-in-range below threshold, and patients meeting targets. Patients with insufficient CGM wear time are identified in a separate category.

Full size image

Perception of less human interaction and workload-related barriers

Based on insights from the interviews, Tidepool-TIDE was designed to increase the fraction of time CDCESs dedicate to patient interactions by reducing their screen time. The major changes made were to reduce the number of tools and processes required for administration and population-management; reduce the time spent accessing data in other CGM platforms; and reduce the time spent switching back-and-forth between the electronic health record (EHR) and TIDE.

We identified administrative and population management needs likely to be necessary at every institution adopting technology-enabled population care: obtaining patient consent and onboarding patients for RPM; assigning clinical reviewers (CDCES) to a cadence of review for each patient; updating patient information; and generating documentation. We incorporated the required functionality into Tidepool-TIDE, thus eliminating the Stanford-specific tools and processes, such as spreadsheets, use of a RedCap database, and ad-hoc management by a research coordinator. For example, Tidepool-TIDE supports and partially automates population management such as adding and removing patients, assigning and updating clinical reviewers, and updating review cadences. To facilitate patient consent and onboarding, Tidepool+ allows clinic staff to import patient accounts from the EHR or create accounts on behalf of their patients. Patients may create an account at the clinic or choose to receive email instructions to create an account from home. Tidepool-TIDE reduces time-consuming switching between TIDE and the EHR to document the care provided and meet the requirements to submit bills for RPM by using the interoperability of Tidepool+ to generate PDF reports for inclusion into the patient chart, on-demand or as an automatic setting, and import CGM summary statistics such as time in range into EHR flowsheets.

The opportunity to most reduce the time required for interventions was eliminating the need for CDCESs to access additional data from hardware manufacturers, primarily pump manufacturers from whom TIDE does not import data and a CGM manufacturer whose web portal offered data visualizations not available in TIDE. Pump data were already incorporated into Tidepool+ and numerous data visualizations not previously available in TIDE were incorporated into Tidepool-TIDE: the Daily View synchronizes all device data over 24 hours, including device setting details at events such as carb entry and basal adjustments; the Device Settings View shows active and saved pump profiles; the Basics View summarizes all device interactions and key endpoints over a variable time period up to 90 days; and an indicator that a patient’s data have been reviewed (Supplementary Figs. 1–3).

Uncertain efficacy in a new population

We designed complementary approaches for potential users of Tidepool-TIDE to estimate the impact of its use before and after deployment. To allow clinics to estimate impact before adoption, Tidepool presents potential users with links to published data around the efficacy of TIDE10,11,12,13,14,15,16. To allow clinics to evaluate the impact of the use of Tidepool-TIDE after adoption we developed functionality that allows randomization as part of routine care, without the additional resources typically required for a clinical trial. The functionality requires neither a research coordinator, nor patients to be randomized out of receiving the standard of care, nor CDCES to dedicate time specifically to the evaluation. In any review period in which fewer of a CDCES’s patients meet criteria for review than their predetermined capacity, a subset of the additional patients that were close to, but did not meet criteria for review, may be added for review. (Fig. 3). This facilitates estimation of the impact of the use of Tidepool-TIDE through comparison of the changes in glucose management between those patients randomly selected for review and those eligible but not selected. There is no change for patients eligible for review, ensuring that all patients receive the standard of care. This approach of micro-randomized control and treatment groups is applicable across clinics since it allows for systematic evaluation of interventions and operational designs (e.g., review cadence, clinic capacity)34,35.

Fig. 3: Patient care flow and partial randomization.
figure 3

Opportunities for leveraging partial randomization (noted by “R”) before and after scaling through Tidepool.

Full size image

Infrastructure and technical barriers

Previously, implementing TIDE in a health institution outside Stanford required in-house information technology (IT) professionals to: develop a pipeline to ingest CGM data and a process for patients to consent to its use; replicate TIDE’s algorithms and visualizations on internal servers; connect their version of TIDE to their internal data sources; set up identity management; and other similar technical and operational support detailed in earlier works describing TIDE as well as reviews of barriers to adopting digital health technology3,4,10,11,12,13,14,15,16. Tidepool is a HIPAA-compliant, SOC-2 Type 2 certified cloud-based application that integrates data from over 85 medical devices and is not deployed locally on a health system server31. Tidepool enters into business associates agreements under HIPAA with all US customers. Institutions may deploy Tidepool-TIDE as a web application, with no support required from institutional IT or servers (Table 2, Fig. 4). Tidepool-TIDE is hosted on Tidepool servers and provides identity management and user interface through a secure web log-in process. For the 1000+ clinics already using Tidepool+ in the US, no additional technical setup is required to launch Tidepool-TIDE; the clinic can reach out to Tidepool sales to gain access to this upgrade. For new clinics adopting Tidepool+ , technical, security, contracting and subscription milestones must first be addressed as part of the standardized onboarding process36. Since Tidepool collects data from numerous diabetes devices, using Tidepool-TIDE for CGMs different from those used at Stanford does not require any additional technical nor operational effort.

Fig. 4: TIDE accessibility beyond Stanford.
figure 4

Technical instrumentation and maintenance by institutional IT professionals are no longer necessary for adoption.

Full size image

Discussion

We described how we developed Tidepool-TIDE, a nationally available, clinical decision-support platform that reproduces the functionality of TIDE, a platform for population-level precision digital health that has been associated with significant equitable improvements to glucose management in youth with type 1 diabetes. Tidepool-TIDE is a health system agnostic platform designed and engineered to reduce common barriers to scaling technology from one site to many. Patient management is supported by a fully interpretable population-level approach to triaging patients and identifying specific opportunities for improved glucose management. No institutional data science, engineering, nor IT professionals are required to set up data pipelines nor to deploy and maintain the platform. Tidepool-TIDE supports core clinical and administrative workflows on a single platform and has the capability to integrate with EMRs, eliminating the need for additional tools and processes to incorporate the RPM-CDS platform into care delivery. Tidepool-TIDE is now available as a turnkey upgrade for over 1000 clinics using Tidepool.

Leveraging technology-enabled care can create opportunities to decrease unnecessary healthcare utilization (e.g., fewer clinical visits for patients with well-managed diabetes) and increase equitable healthcare access (e.g., patients’ condition can be remotely monitored and managed regardless of geographical location or other factors that may pose challenges to clinical visits). A key principle of the 4T study has been equitable access for all newly diagnosed youth with T1D. Data from the 4T Pilot study demonstrated similar improvement in HbA1c at 1-year after diagnosis independent of insurance or language status12. The technology-enabled care model may serve as an example for equitable, effective, and efficient management of other health conditions such as heart disease and asthma, where remote patient monitoring has been beneficial37. For example, a Tidepool-TIDE-like platform connected to wearable devices and sensors measuring vitals, physical activity, and inhaler usage may help clinicians identify and care for patients at risk of deteriorating cardiovascular or respiratory functions at population scale38.

The increased adoption of CDS for RPM has come with increased concerns about its efficacy and cost effectiveness. While randomized controlled trials are the gold standard for evaluating the efficacy of a new intervention, incorporating them into clinical care pose significant costs, challenges, and the ethical dilemma that a subset of patients is excluded from the potential benefits of the technology39,40. These concerns can be minimized while ensuring internal validity through partial randomization. Outside of healthcare, it is common practice for companies to employ randomized A/B testing to compare different options, evaluate effectiveness of iterative changes, and make data-driven decisions before deploying something to the entire audience41,42. However, health systems tend to implement interventions wholesale and rarely formally analyze the effectiveness of interventions deployed43. We developed functionality to allow partial randomization as part of routine care to facilitate larger research studies across health institutions and systematically measure the impact of any changes to care delivery. Partial randomization allows Tidepool-TIDE to continue to present all patients that meet criteria for review while introducing an aspect of randomization for inclusion of additional patients who don’t meet the criteria for review. This has the potential to facilitate rigorous evaluation of the impact of the use of Tidepool-TIDE and of any iterative changes to clinical interventions or operational design through micro-randomized control and treatment groups. Since the number of patients meeting the criteria for review may vary from review period to review period, our proposed randomization method would include additional patients only in those periods where there is sufficient capacity for them. Numerous statistical design and analysis methods are now available to leverage the partial randomization functionality enabled by platforms such as Tidepool-TIDE34,35,44. The combination of the partial randomization functionality of Tidepool-TIDE together with appropriate statistical design and analysis has the potential to enable a virtuous feedback cycle for continuous improvement. The use of partial randomization as part of routine care made possible in Tidepool-TIDE will require additional study and site-specific considerations to ensure patient safety and valid experimental design.

Tidepool-TIDE is now in use at Stanford Children’s Health to support patient care for the 4T study in a pilot phase, facilitating continuous evaluation and improvement through survey feedback from the clinical team and proxies to time-per-patient-review. Following the pilot phase, the goal is to transition this research care model into Stanford Children’s standard of care so that all pediatric T1D patients are reviewed through Tidepool-TIDE. Data collection for a formal evaluation of the ease-of-use and clinical efficacy of Tidepool-TIDE compared to TIDE is ongoing.

A component of the transition to Tidepool-TIDE has been to collaborate with the hospital’s billing and compliance teams to create financially supported and sustainable RPM workflows. While RPM evaluation and management codes were included in the 2018 Centers for Medicare and Medicaid Services fee schedule, payment for these RPM codes vary state-by-state37,45. Some codes are also focused on adult populations, limiting pediatric usage of RPM codes45. For self-sustaining and equitable RPM implementation at scale, consistent health plan reimbursement, especially for public plans like Medicaid is critical15. This is an ongoing topic of investigation.

Beyond Stanford, we are collaborating with other diabetes centers in the US to develop research studies to disseminate and implement the 4T care model powered by Tidepool-TIDE. Access to the Tidepool-TIDE platform will provide opportunities for other clinics and investigators to innovate on how to apply RPM to improve care delivery and outcomes in their healthcare systems. This can be particularly impactful as sample size has thus far been a main limitation of pediatric telemedicine studies2. Future work should adopt successful methodologies and guiding principles from the study of innovation. For example, classifying diabetes centers by their attitudes to the adoption of new technology may facilitate identifying promising partners to scale the use of Tidepool-TIDE46.

We prioritized addressing user needs that were fundamental for effective and efficient technology-enabled care, differentiating them from health institution-specific or study-specific needs that may be an artifact of having initially launched the program at Stanford. These features were implemented in the initial version of Tidepool-TIDE while others were scoped for a future iteration. Future iterations of Tidepool-TIDE can incorporate data from more wearable devices for a richer view into patient conditions and explore artificial intelligence (AI)-based approaches that can provide more sophisticated insights for care management than a rules-based approach. Data from additional sources such as activity monitors can enhance existing algorithms, accounting for how other comorbidities may influence T1D management and outcomes47. AI-generated recommendations can also be incorporated into provider-patient messaging to improve patient education and promote patient engagement. Although Tidepool-TIDE is exclusively designed for diabetes care management, our approach may offer lessons for institutions interested in making their locally developed digital tools for various chronic health conditions more broadly accessible.

The primary limitation of this work is that data are not yet available on the user experience nor patient outcomes associated with the transition from TIDE to Tidepool-TIDE at Stanford, from deployments of Tidepool-TIDE at other institutions, or to other platforms. However, as most patients and clinics are not cared for with a CDS platform and TIDE is an integral part of the 4T study which showed a 1.1% decrease in A1c, we are optimistic that the useability and outcomes associated with the use of Tidepool-TIDE, which replicates and improves on TIDE, will compare favorably to the status quo12,13,14,15,16,18. As this was a large software development project, some technical details necessary for the reproducibility of the paper are missing or summarized. Since Tidepool is open-source, the entirety of the implementation and code base are available for review48. Whether the Tidepool-TIDE platform may improve outcomes independent of the overall 4T diabetes care management approach has not been tested. Unlike insulin dosing CDS tools, Tidepool-TIDE currently does not formulate and communicate recommendations directly with patients or caregivers12. Rather, it automates data analysis at population-scale so that clinicians can prioritize their attention on patients with deteriorating diabetes conditions and perform timely clinical interventions orthogonal to in-person visits. Similar decision support features could be implemented within the Tidepool-TIDE model in the future.

RPM-CDS platforms like Tidepool-TIDE depend on access to electronic health information (EHI). Prior to the 21st Century Cures Act, information blocking was a prevalent issue in healthcare innovation as EHR vendors and health systems partook in practice “likely to interfere with, prevent, or materially discourage access, exchange, or use of electronic health information”49,50,51. While the passage of the Cures Act mandating healthcare organizations to share EHI and meet standards of interoperability is promising, the Cures Act does not impose the same requirements on medical device manufacturers, which can pose challenges to access of data collected through wearable devices51,52. Data access is subject to the individual cooperativity of device manufacturers, their internal prioritization of the feature against other roadmap features that could be more directly tied to revenue generation, and the speed with which they are able to support data sharing for devices already released on the market. Patients and clinicians may go months or years without having access to their diabetes EHI. Public policy that includes device manufacturers in the standards of interoperability and data sharing may increase data access to facilitate better care and health delivery innovation.

Using design principles, workflow analysis, and software development, an interdisciplinary team from Stanford and Tidepool developed Tidepool-TIDE, a clinic-agnostic, nationally-available tool for personalized diabetes care at population scale. Tidepool-TIDE reproduces and improves the core functionality of the original tool that, when used as part of the 4T Study, was associated with significant, equitable reductions in patient HbA1c and reductions in care provider screen time. Tidepool-TIDE is now available as a turnkey option for over 1000 US clinics using Tidepool.

Methods

Background

The original TIDE platform prioritizes T1D patients for review and potential contact by care providers. TIDE does so with an algorithm that analyzes CGM glucose values based on ATTD consensus CGM glucose targets and a personalized measure of change in time in ranges33. The TIDE algorithms had been optimized for sensitivity and specificity at the Stanford Children’s endocrinology clinics10,16. Based on clinician guidance, the algorithm takes a rule-based approach as opposed to using machine learning or time-series analysis10. While effective, the original TIDE did not directly support all administrative and clinical workflows in one place, resulting in clinicians and coordinators relying on additional tools and adjacent processes to coordinate and deliver care such as use of spreadsheets to manage patient panels. Some elements of TIDE’s evolution were also Stanford-specific and replication of TIDE at other institutions would require adjustments and redeployment. For example, data pipelines would need to be re-created and alerting metrics may need to be adjusted to fit the characteristics of each individual clinic16. TIDE is part of Stanford’s 4T study, which was approved by the Stanford Institutional Review Board and informed consent (and assent for participants aged 7-18 years) was obtained from all participants. The 4T study is registered on ClinicalTrials.gov as NCT03968055 (registered on 5/28/19) and NCT04336969 (registered on 4/3/20).

User research

To address these limitations, 4T and Tidepool engineers, designers, and product managers collaborated to design a new version of TIDE as part of the Tidepool platform (Tidepool-TIDE) that could be as effective as the original tool while also being more comprehensive, scalable, and health-system agnostic. To identify core user needs, the team conducted user research with CDCES who use TIDE to help patients manage T1D, as well as with coordinators who manage the administrative workflows. We conducted a series of stakeholder interviews, shadowed clinicians, and held a paper-prototyping workshop with the CDCES team. Our UXR focused specifically on “clinical workarounds”: places where CDCES’ created their own processes on top of TIDE to provide personalized care to patients, or to make up for perceived inequities or biases within the healthcare system.

Platform development

Our interdisciplinary team met regularly over 2–3 months to iterate on designs for building a population care management tool into Tidepool+ that would be compatible with the existing Tidepool technical architecture and minimize engineering complexity. Engineers shared documentation and open-source Tidepool and TIDE code to build shared context around existing and future technical implementation pathways.

We analyzed all aspects of patient onboarding, education, and tracking to separate the aspects that would need to be reproduced for other clinics from those that were specific to the original center and study. In discussing Tidepool’s onboarding and integration processes, we identified which workflows could be reused and leveraged to reduce friction health systems face in adopting this technology-enabled care model.

We brainstormed technical features that would be beneficial in a broader deployment, such as ways to support wider-scale, dynamic informatics studies that were not available in the original deployment of TIDE. We explored options considering Tidepool’s current technical infrastructure.