Introduction

Research has an enormous influence on the effectiveness of health care and in many ways defines the future of public health1. Success and failure of research are no longer just academic matters but also impact health professionals and relevant to policy making2.

Biomedical research benefits public health improvement through landmark scientific discoveries in the long term and also through association with higher-quality care and better health outcomes in the short term3. The NIH Cancer Centers Program recognizes centers that meet high standards for transdisciplinary, state-of-the-art research on preventing, diagnosing, and treating cancer4. Numerous data-driven studies documented superior healthcare outcomes in National Cancer Institute designated Comprehensive Cancer Centers5,6,7,8,9. A similarly positive impact of biomedical research culture on care quality has also been shown in other settings and broader, population-based studies10,11.

Many other observations also urge expansion and improvement in biomedical research. Developing a new drug takes well over a decade and has a failure rate of more than 95%. The most expensive failures are in late-phase clinical trials. Therefore, it is pressing need to better pinpoint the right biological targets, like genes, proteins, and other molecules, early in the research process12. Furthermore, about 7,000–10,000 rare diseases affect more than 300 million people worldwide and 10% of the US population13. Unfortunately, there are approved drugs for only about 5% of these rare diseases14.

While actual research growth largely happens at specific universities and specialized institutions, most studies of growth are focused on a higher, regional or national level. For example, some analyzers estimated that an annual increase in the overall NIH budget is essential to maintain the momentum of progress in the battle against diseases15. Life sciences research accounted for more than half of total R&D spending in 202116. Between 2002 and 2022, the number of U.S. life sciences researchers grew by 87%, compared to only 14% for all U.S. occupations17.

Our earlier studies showed that research performance dominates the international ranking of universities18. Half of ranking systems are fully reliant on research indicators and the rest is also dominated by research. A large variety of measures have been proposed or actually used as institutional performance indicators of research performance. Among them one of the most prominently applied measures is competitive grant funding received from extramural sources. It is used by the Carnegie Classification of Institutions of Higher Education and also by this study. While more complex and multiple measures could be appealing, the varying availability of data and frequent reliance on self-reporting make them less trusted. Meanwhile, the feasibility and informativeness of best practice comparison studies demonstrated that biomedical research leading to landmark discoveries has trajectories identifiable by big data analyses19.

Only few other studies provide indirect information about institutional factors of successful research growth. Most available institution level studies tend to assess research productivity by frequently used bibliometric measures such as number of publications or number of citations20. A study of faculty workloads found that more than two-fifths of time allocated to federally funded research was spent on pre- and post-award administrative activities rather than active research, with variations across institutions and disciplines21. Economic impact studies show that about 10% of NIH grants generate a patent directly but 30% generate articles that are cited in patents afterwards22.

While research success is an elusive concept, becoming more and more competitive in getting NIH grants is a widely accepted performance indicator that lays the foundation of further achievements, like funding from other sources, publications, patents, and beneficial discoveries. The purpose of this study was to address the gap of understanding by focusing on factors influencing progress at the level of universities and research institutions and to uncover characteristics of the research environments that promote fastest growth and investigator success in these organizations.

Results

Out of 2648 NIH funded US organizations, the resulting total sample included 254 eligible NIH supported research universities and organizations (Table 1). This sample had 164 Carnegie classified universities and 90 non-Carnegie research organizations (i.e., independent, non-educational research entities and institutes).

Table 1 Baseline comparison of fast grower and control research organizations according to Carnegie classification (data from the NIH RePORTER and RePORT between fiscal years 2012–2021).
Full size table

Baseline comparison of the fast growers and control research organizations according to Carnegie classification categories shows statistically significant differences. The number of Carnegie R1 universities in the fast grower group is nearly double those in the control group, while R2 universities and non-Carnegie institutions are more than double in the control group. Some Carnegie classification categories were only represented in the not fast growing, control group (e.g., Special Focus Four-Year Other Health Professions Schools, Master’s Colleges & Universities Larger Programs).

At the baseline of observation period, the two groups were comparable in terms of grant funded project performance. Only marginal differences were observed between fast growers and controls. One performance measure, the initial NIH funding per PI (M$) reached the level of significance required by our study (Table 1).

At the end of the observation period, several major differences can be observed not just in terms of percentage growth as expected (last year divided by first year) but also in terms of various PI ratios related to the initial number of PIs. The fast growers show a statistically significantly higher growth in the number of PIs, awards, funding per PI, and awards per PIs (Table 2). The fast growers group also outperforms the control group regarding the size of the talent pool of PIs, continuously active PIs, recruited PIs, and internally raised PIs during the ten-year period.

Table 2 Performance comparison of fast grower and control research organizations based on NIH supported research at the end of the 2011–2020 period (NIH RePORTER and RePORT; 2012–2021).
Full size table

In the ranked list of top 25 fast growing research institutions, the majority are non-Carnegie research organizations followed by a few Carnegie R1 Universities based on their progression of NIH funding. There are only two Carnegie R2 universities in this elite group of fast growers.

In time series comparison, the percentage of PIs regaining funding is significantly greater in the fast grower group than in the control group every subsequent year after a funded year (x = 27.7363, P < 0.001). The chance of regaining funding in subsequent years is 76% in fast growers and 75% in control organizations. Similarly, 8.9% of fast growers and 8.4% of the controls had funding after one year gap of no funding; 3.6% of the fast growers and 3.1% of the control group PIs regained funding after a two-year gap; and 1.8% of fast growers and 1.6% of the control group PIs received funding after a three-year gap. The chance of returning to funded PI status after an unfunded year within 8 years is 29.75% in fast growing research organizations but it is only 26.44% in control organizations.

The talent pool of researchers included every NIH funded principal investigators, who had been employed in the particular organization for any period of time and who had not left the organization during the analyzed 10 year. Consequently, the final year of analyzed 10 years can be characterized by the number of actively founded principal investigators and by the number of non-active researchers in the talent pool who are employed but not grant funded PIs in the final year. The median ratio of these two numbers is 1.33 (IQR: 0.54). Results of the Spearman rank correlation indicated a significant large negative relationship between the percentage growth in NIH funded research and the ratio of non-active versus active principal investigators (r(s) = -0.5514, p = 1.318e-21).

Figure 1a shows the trend of NIH funding growth in the two groups respectively. It indicates that the amount of funding in the fast grower group has been steadily increasing over time, while the control group has remained largely stationary. At the beginning of the ten-year period the highest funded 25% of organizations received 18 times more NIH funding than the lowest 25%. Ten years later, the highest funded 25% received 30 times more NIH funding than the lowest 25% in the analyzed group. In other words, the grant funding disparities are rapidly growing. The relationship between patterns of growth for six actionable measures and the resulting increase of amount of funding for the study groups is illustrated by the radar charts in Figs. 1b and Fig. 2a-d.

Fig. 1
Fig. 1
Full size image

Overall trends of research growth and performance radar charts of institutional research growth strategies (fast growers: red, control group: blue): (a) Trend of growth in the fast grower and control group based on the amount of NIH funding (2012-2021); (b) Comparison between fast grower and control research organizations based on six actionable measures and the resulting NIH funding increases (2012- 2021).

Fig. 2
Fig. 2
Full size image

Illustrative performance radar charts of selected organizations based on six actionable measures and the resulting NIH funding increases: (a) well-balanced fast grower research university doing well by most measures; (b) fast grower, non-Carnegie research organization showing a large jump in talent pool by internally raised new investigators ; (c) Carnegie fast grower research university showing major increase in the average grant budget size per PI; and (d) non-Carnegie fast grower research organization showing both new PIs and major increase in the average grant budget size per PI (Based on data from the NIH RePORTER and RePORT databases between fiscal years 2012–2021).

The growth of a research organization can be considered well-balanced when it is doing equally well in most action lines as opposed to relying only on one action line in competing with others. Apparently, universities and institutions of the fast grower group can be better in all performance measures but there are some visible strategic variations (i.e., balanced improvement in many measures, large jump in talent pool, major increase in the average grant budget size per PI, increase in new PIs and also per PI funding). The fast grower organizations develop twice as large talent pool over 10 years.

Discussion

Sustained high growth rate of NIH funding appears to be a valuable indicator of more effective research support culture. Grant funding of the fastest growing organizations prominently outpaces the growth of the NIH budget over the same period. While R1 universities and non-Carnegie research institutes are among the fastest growing research organizations, neither current NIH funding levels nor academic classifications of organizations appear to be useful identifiers of a supportive research culture.

Our list of measures having significant impact on growth of grant funding highlights important and actionable opportunities for developing a highly productive research culture. In the final year talent pool calculation, we found 13 non-active PIs for every 10 actively funded PIs and this ratio is even larger in non-growing organizations. The significant negative correlation further underscores that successfully growing research organizations are also successful in mobilizing their talent pool of researchers and in turning non-active but experienced principal investigators into actively funded principal investigators.

In summary, the largest contributors of fast growth are supporting researchers who lost funding to gain funding again and to raise unfunded faculty into funded status. Intramural raising of PIs appears to be a more important source of new grant funding than recruitment of funded PIs.

This data driven quantitative analysis offers unique insight into the most informative institutional characteristics of fast growth in research funding. The findings revealed, there is no silver bullet for promoting fast growth of funded research. Instead, combination of multiple measures can make the difference. In other words, sustained high growth rate is the result of numerous improvements and actions.

Our results underscore that organizational culture, especially wide-ranging support for focus and persistence, is essential in expanding biomedical research at universities and research organizations. Currently, many principal investigators are One-and-Done researchers. For example, 43% of new Principal Investigators fail to compete successfully for further funding23. Similarly, 53% of Principal investigators conduct only one clinical trial and never return24. In studies of human resources, it is widely recognized that high employee turnover is a significant cost for businesses, increases operational uncertainty, and is negatively associated with future financial performance25. Therefore, research universities and organizations need to increase the ratio of continuously high-achieving researchers and reduce the turnover of Principal Investigators.

Principal investigators should also strive to develop research leadership and successive project management skills besides looking for supportive organizational culture. Studies of Nobel Laureates show the extraordinary value of sustained commitment, successive projects, and thematic research26,27. The managerial challenges experienced by publicly funded Principal Investigators is also attracting increased attention28,29,30. As a result, development of research leadership is becoming included in a few training programs31. However, much greater and broader research leadership development would be needed to achieve substantial increase in research productivity and new discoveries.

By focusing on biomedical research, this study relied on data derived from the NIH grant award database. Therefore, our results could not be easily generalized to other countries. In the US, there are many other federal and private funding sources like NSF, CDC, SAMHSA, AHRQ, HRSA, RWJF, or HHMI. However, relative to NIH these funding sources play a smaller role in supporting life sciences research. Due to lack of sufficiently detailed, comparable data sources, our methodology of identifying employment stories of principal investigators of research projects could not be consistently applicable to these other funding sources.

The RePORTER and RePORT databases are often used and generally viewed as high quality. However, some errors were discovered (e.g., multiple IDs of the same PI; incorrect tracking of some university mergers). Fortunately, these errors were exceedingly rare, transparently handled by our curation protocol, and unlikely to influence the results of this study. Finally, this study did not investigate individual researchers’ perspectives or behaviors which may provide a deeper, more detailed understanding of the different circumstances of growth.

In moving forward, ambitious researchers should consider not only their own scientific interests and priorities but also expectations of productive and supportive research culture of their employer. The productivity of research laboratories depends more on a range of organizational culture factors than generally assumed. High performing organizations need to draw upon existing faculty strengths and encourage experienced researchers to take on new research projects.

It is recommended that research leaders of universities and institutions facilitate the development of individual researchers, demonstrate a strong commitment to skill development, and focus on the entire talent pool, not just on the currently funded investigators. Responsible leaders at universities and research institutes may benefit from better understanding of the measurable factors that play a significant role in the quality and productivity of research culture.

Methods

This best practice analysis focused on the research performance of institutions based on their grant-funded research and their NIH funding growth over a ten-year period.

Sample

Eligibility criteria of the study sample of US based universities and research institutes included (i) received NIH funding both in 2012 and 2021; (ii) their name and identification number (Id) remained identical throughout the decade; and (iii) had an initial amount of NIH funding at least $10 M due to the different dynamics of smaller organizations.

Data source

Relevant data were retrieved from the NIH RePORTER and RePORT databases between fiscal years 2012–2021. After identification of sample organizations, the retrieval and analyses included all NIH funded US based researchers affiliated as contact Principal Investigators (PIs) with these organizations for one or more years at any time during the analyzed ten years. For degree-granting institutions/universities, the Universal classifications were retrieved from the Carnegie Classification of Institutions of Higher Education.

Variables

Lifetime funding information was collected about all Contact PIs affiliated with the sample organizations. It included NIH Spending Categorization, Project Title, Project Number, Subproject Number, Contact PI Person ID, Contact PI / Project Leader, Organization ID (IPF), Organization Name, Organization City, Organization State, Organization Type, Organization Zip, Funding Mechanism, Fiscal Year, Total Cost, Total Cost (Sub Projects), Funding IC(s), Direct Cost IC, Indirect Cost IC, Total Cost IC.

Data retrieval

After the selection of samples, the award records for each target organization for the 10-year interval were exported. As a crosscheck, any PIs who have never been affiliated with the target organization but still appeared in the exported data were excluded. After creating a list of unduplicated affiliated PIs in each organization, all PIs complete award records were downloaded from NIH RePORTER regardless of affiliation changes during the 10-year interval. Finally, the exported files for each organization were collected in one file and the data for all PIs from all organizations were also collected together.

Data processing

This study used structured database, purpose-built algorithms, and commercially available statistical programs.

Sample organizations were ranked based on percent growth between 2012 and 2021. Unlike absolute growth, percentage growth of funding characterizes the dynamics of growth regardless of institution size. After calculating the percentage funding increase/decrease over the above-defined decade, the sample was split into the group of fast-growing research institutes and an equally sized control group of institutions.

Two purpose built algorithms and programs were developed to calculate employment and award performance data for each included PIs and sample institutions: Annual statistics (Number of PIs in the target organization, Number of projects received, Amount of funding received ($M), Efficiency - Projects per PI, Efficiency - Amount per PI, Number of formerly active but now inactive PIs; Changes in the PI pool (PI number change; Recruited, Lost, Internally raised PIs); Performance measures (PIs having X year of funding, Talent pool of PIs, % growth of number of actively funded PIs, % growth of projects per PI, % growth of amount of funding, % growth of funding per PI, % Continuously active PIs, % of recruited PIs, % of retained PIs, % of internally raised PIs). Internally raised PI number counts researchers who were unfunded before but become newly funded as contact Principal Investigators.

Quality control

Collected data were quality checked and appropriately matched based on the following principles:

  • The number of PIs and the total amount of funding from the NIH RePORT was compared with the downloaded number of PIs and funding records from NIH RePORTER in starting and ending years to assure completeness.

  • In very rare cases, the same PI had several identification numbers and these were left in the list to assure completeness.

  • A program was developed to select variables (columns) for each specific analysis. All non-used variables (i.e., unnecessary columns) were removed in preparation for a particular analysis.

  • Comparisons of the number of PIs used an employment table calculation with the number of PIs in each target organization for every year.

Statistical analysis

The curated and processed productivity tables were used for portrayal of trends over time and for clustering of organizations based on their Carnegie classification and other comparisons.

The median and interquartile range were calculated for each variable. Due to the number of statistical tests conducted in this study, we set the threshold of significance to < 0.005 to control the level of Type I error.

Mann-Whitney test was used to compare the study groups regarding most calculated variables. Comparisons of distribution of researchers between the two study groups were made by Kolmogorov-Smirnov test. It was also used to test the difference in distribution of researchers in various Carnegie groups. Chi-square was used to test the comparison between the study groups regarding chance of PIs to be funded again after one or more gap years.