Introduction

Scientific and technological (S&T) innovation serves as a strategic pillar for enhancing social productivity and national strength1. Currently, nations around the globe place significant emphasis on S&T innovation, with a new wave of technological revolution and industrial transformation unfolding, accompanied by increasingly fierce competition for technological leadership. The United Nations Sustainable Development Summit, held in 2023, introduced the Technology Enabler Mechanism, aimed at promoting the advancement of science, technology, and innovation to support the achievement of the 2030 Agenda for Sustainable Development. The further integration of S&T innovation capabilities, coupled with the optimization of resource allocation, is expected to significantly enhance the overall performance of the national innovation system. This, in turn, will not only bolster national S&T innovation capacity but also play a critical role in fostering and developing new forms of productive forces.

S&T innovation is fundamental to the advancement of scientific research. By improving innovation effectiveness, research institutions and researchers can more swiftly understand and resolve complex problems, thereby facilitating the deeper application and value enhancement of scientific knowledge2. High-quality S&T innovation drives the translation of research outcomes and serve as a bridge between basic and applied research. For researchers, research institutions, and research management organizations, enhancing the effectiveness of S&T innovation is a prerequisite for the successful generation, effective transformation, and in-depth application of research outcomes.

Regarding the enhancement of S&T innovation effectiveness, the academic community generally views it as a reflection of the scientific and rational formulation and implementation of science and technology policies3; On the other hand, it is considered a positive indication of researchers’ awareness and initiative in S&T innovation4. In recent years, to meet the objectives and practical demands of S&T development, governments at the national, provincial, and local levels, along with various research institutions, have continuously optimized and refined science and technology policies5. These improvements encompass areas such as the utilization of R&D funding, the establishment of technological platforms and innovation bases, the provision and sharing of technological resources, the protection of technological achievements and intellectual property rights, and the recruitment and incentivization of scientific talent. These efforts aim to better achieve the comprehensive goal of advancing S&T innovation and high-quality development.

China has emphasized the need for ongoing reform in the field of science and technology. Both S&T innovation and institutional innovation must function together, acting as interdependent drivers of progress. However, challenges still remain in enhancing the effectiveness of S&T innovation in China, including policy delays, weak implementation, systemic and institutional barriers, and poor policy coordination6. In recent years, China’s scientific research institutions have made significant strides in developing both hard and soft environments, leading to a steady increase in their innovation efficiency. However, they still fall short compared to innovative countries like the United States, the United Kingdom, and Japan7,8. Relevant scholars have developed measurement indicators and evaluation systems for S&T innovation efficiency, taking into account regional differences (such as the central region and the Yangtze River Economic Belt), industrial sectors (such as agriculture and marine), and primary fields (such as universities, research and development institutions). Nevertheless, existing research has yet to systematically tackle the key question of which factors influence the enhancement of S&T innovation efficiency from the perspective of S&T researchers’ perception.

So in this context, several key issues should be required more attention: What are the critical factors influencing the enhancement of S&T innovation effectiveness? How can we pinpoint breakthrough points and focal areas for improving S&T innovation effectiveness and scientifically formulate and optimize science and technology policies? How can we better support S&T innovation and further stimulate innovative enthusiasm, ultimately achieving the intended goals of technological innovation?

On the basis of the above issues, it is essential to analyze the critical factors that impact the enhancement of S&T innovation effectiveness. Scientific researchers are the most fundamental agents in the S&T innovation process and the primary evaluators of its effectiveness. Given this, the study employs survey interviews and Grounded Theory (GT) methodology to identify the critical factors influencing the enhancement of S&T innovation effectiveness from the perspective of researchers within scientific research institutions, primarily universities and research institutes. The aim is to uncover solutions to challenges such as institutional inertia and policy barriers in S&T innovation, thereby providing valuable insights for the further optimization of science and technology policies and enhancing overall innovation effectiveness.

The remaining structure of this study is organized as follows: The second section provides a literature review. The third section outlines the research design of the study. The fourth section presents the construction and test of the critical factor model based on GT. The fifth section focuses on the development of the measurement scale and the identification of the critical factors’ importance. The sixth section discusses the importance of the critical factors, offers relevant policy recommendations, and elaborates on the research implications. The final section concludes the article, highlighting the research limitations and proposing directions for future research.

Theoretical foundations

Since the 18th National Congress, China has increasingly prioritized on enhancing the effectiveness of input and output in S&T innovation. The “13th Five-Year Plan for National Science and Technology Innovation”, introduced in 2015, emphasized the importance of increasing investment in S&T innovation and improving the conversion rate of research outcomes. In 2022, the report from the 20th National Congress reaffirmed the significance of strengthening the S&T innovation system, underscoring innovation as the core driver of national modernization. Furthermore, in 2024, the Third Plenary Session of the 20th Central Committee highlighted the necessity of fully implementing the strategies of invigorating the country through science and education, strengthening the nation through talent, and fostering innovation-driven development. These strategies aim to enhance the overall effectiveness of the national innovation system.

The role of science and technology policies in enhancing S&T innovation effectiveness

In the early stages of implementating of science and technology innovation policies, scholars primarily focused on the incentive effects of these policies on innovation effectiveness. Wang et al.9 highlighted that sustainable public procurement serves as a powerful tool for promoting S&T innovation effectiveness. Using policy bibliometrics, they analyzed China’s public procurement policies aimed at fostering S&T innovation, revealed the decision-making characteristics. Wang et al.10 conducted an empirical analysis, demonstrating that government tax credit policies can boost R&D investments and patent applications in technology firms, thereby enhancing S&T innovation effectiveness. Similarly, Wan et al.11, applying the SAR Tobit econometric model, found that tax incentives significantly crowd in R&D technological efficiency, influencing firms’ overall S&T innovation effectiveness. Wang et al.12 employed linear regression and qualitative comparative analysis (QCA) to evaluate the impact of innovation policies on the S&T innovation effectiveness of high-tech industrial parks. They concluded that demand-driven policy tools have the most substantial impact, while supply-and environment-based tools exert positive effects during policy lag periods. Chen et al.13 verified the role of the policy environment on knowledge transfer through an econometric model, finding that knowledge transfer significantly enhances the effectiveness of S&T innovation.

The measurement and evaluation of the effectiveness of S&T innovation

As research on S&T innovation effectiveness deepens, the focus has gradually shifted towards measuring the level of S&T innovation effectiveness. Wang et al.14 evaluated the effectiveness of S&T innovation in China by examining three resource dimensions: technology, funding, and talent. They selected input indicators such as the number of science and technology personnel, the proportion of research funding in government expenditure, and the number of internet users. Output indicators included the number of scientific papers, patent applications, and the contribution rate of technological outputs. Building on this framework, Zhang et al.15 further classified S&T innovation into five types: foundational capabilities, technological innovation environment, production capabilities, research capabilities, and transformation capabilities. They developed a comprehensive measurement system for S&T innovation effectiveness, consisting of 9 primary indicators and 22 secondary indicators. Zuo et al.16 refined the single-stage input–output process into a two-stage input–output model. They proposed a two-stage DEA model to evaluate the S&T innovation effectiveness of the mining industry across 30 Chinese provinces from 2008 to 2018, incorporating both initial and additional intermediate inputs as well as final and free intermediate outputs. Chen et al.17 developed an indicator system to evaluate the S&T innovation effectiveness of China’s world-class universities. The input indicator system comprises two secondary indicators: “human resources” and “research funding”, while the output system includes three secondary indicators: “university basic S&T innovation”, “university applied S&T innovation”, and “university experimental and development innovation”.

The investigation of critical factors influencing the enhancement of S&T innovation effectiveness

Furthermore, several scholars have conducted empirical studies to explore critical factors influencing the effectiveness of S&T innovation. Chen et al.18 applied a spatial econometrics model to analyze factors affecting the effectiveness of S&T innovation in high-tech industries. Their findings revealed that government support, R&D input, industry agglomeration, economic openness, and modern service industry development all significantly impact innovation effectiveness. Luo19 utilized the DEA method to measure S&T innovation effectiveness in Chinese universities and identified critical factors such as economic advantages, geographic location, government support, and research infrastructure. Wang et al.7 emphasized that technological innovation efficiency (TIE) is a critical indicator and driving factor for assessing the balance between innovation inputs and outputs, noting that China’s innovation efficiency is significantly lower than that of other developed nations. Ahn et al.20 applied the resource-based view (RBV) to confirm the impact of S&T capabilities and innovation spirit on S&T innovation effectiveness through ordinary least squares (OLS) regression. Wijaya et al.21 identified five critical factors influencing S&T innovation effectiveness: knowledge absorption and external resources, organizational design and cultural framework, practices and communication dynamics, individual creativity and innovation potential, and organizational learning environment. Han et al.22 conducted a categorical analysis of the coupling and coordination between well-being and innovation effectiveness, revealing a significant positive correlation between human well-being and innovation effectiveness.

In summary, based on the research literature and recent developments, it is evident that studies on the effectiveness of S&T innovation in China primarily focus on policy incentives and resource allocation as central themes. The research has progressively evolved from “macro policy analysis” to the “construction of effectiveness measurement systems” and finally to the “identification of key elements”. Early studies mainly concentrated on policy incentive mechanisms, while mid-term research shifted toward exploring efficiency measurement methods, such as input–output frameworks and system dynamics models. In recent years, attention has increasingly turned to the micro-level driving mechanisms of S&T innovation efficiency, such as the adaptation pathways of S&T resources.

The main findings can be summarized as follows: First, science and technology policy plays a crucial role in enhancing the efficiency of S&T innovation. For instance, innovation output is directly stimulated by R&D subsidies and tax incentives8, although the implementation and coordination of talent policies remain insufficient23. Second, regarding the measurement of S&T innovation efficiency, most studies primarily use scientific and technical journal papers24, patents25, and technology transfer income26 as output indicators, but often overlook the nonlinear effects of tacit knowledge flow and team collaboration on efficiency.

The theoretical gaps in relevant research are as follows: First, there is a lack of the subject perspective. Most existing studies are conducted from the viewpoint of policymakers or enterprises27,28, but they pay insufficient attention to the micro-level perceptions of scientific researchers, who are the core subjects of S&T innovation11. This is especially true regarding the in-depth analysis of their internal innovation motivations, such as research missions and willingness to cooperate. Second, there is fragmentation in mechanism research. Key factors, such as team collaboration and policy incentives, that improve the efficiency of S&T innovation are mostly analyzed in isolation, lacking a systematic dynamic model that integrates “individual capability, team collaboration, policy support, and environmental adaptation”. Third, there are limitations in research methodology. Traditional quantitative research mainly relies on explicit data, such as R&D investment and patent counts29, making it difficult to capture the nonlinear characteristics of the innovation process, including the contingency of disruptive technological breakthroughs. Meanwhile, case studies and qualitative analyses often focus on specific regions or industries30,31, lacking universal conclusions across domains and long-term comparisons. Additionally, existing research is mainly confined to management frameworks and fails to fully incorporate perspectives from psychology (internal motivation theory), sociology (scientific community theory), and other disciplines.

Existing research has achieved a fundamental consensus on policy-driven approaches and efficiency measurement; however, the singularity of theoretical perspectives and the rigidity of methodologies restrict the deconstruction of the complex mechanisms underlying the effectiveness of S&T innovation. It is essential to focus on and analyze the internal mechanisms of both subjective and objective factors to improve the efficiency of S&T innovation from the primary perspective of research institutions and scientific personnel, thereby enhancing the explanatory power of theories and the adaptability of policies.

Therefore, this study starts with researchers’ perceptions of S&T innovation and applies GT to explore the critical factors and theoretical framework influencing the enhancement of innovation effectiveness. Additionally, the study empirically analyzes these critical factors using measurement scales, providing recommendations for the formulation and optimization of science and technology policies.

The potential marginal contributions of this study are as follows: (1) it adopts the perspective of researchers—who are the primary agents in implementing S&T innovation effectiveness—and uses GT to identify the critical factors influencing innovation effectiveness, thereby providing a more comprehensive view for policy research on innovation effectiveness; (2) based on the qualitative analysis and coding results from GT, the study designs and administers a survey to validate the critical factors perceived by researchers, aiming to identify the focus and mechanisms of policies for improving innovation effectiveness. This approach helps address gaps in existing research and enriches the theoretical framework of technological innovation reform.

Research design

Investigating the critical factors that influence the enhancement of S&T innovation effectiveness provides important reference points for research management organizations in formulating and refining science and technology policies. Additionally, it aids research institutions identify weaknesses and focal points in research organization and management, while also reflecting the concerns and genuine perceptions of researchers regarding S&T innovation. Therefore, conducting this study is of significant importance and value.

To enhance the clarity of this study, the research framework is presented in Fig. 1.

Fig. 1
figure 1

Research framework.

Full size image

Selection of research methodology

Grounded theory (GT), as a qualitative research method, primarily involves exploring and inductively analyzing a specific issue or phenomenon32. With its distinctive approach to theoretical development and data analysis, GT is widely utilized across various disciplines, particularly in exploring the underlying causes and mechanisms of complex social phenomena33. The data sources for GT can encompass interviews, surveys, policy documents, online information, and academic literature, all of which are characterized by their breadth and richness34. Thus, the selection of GT as the methodological framework for this study aims to systematically uncover and conceptualize the core characteristics and patterns of the research subject through a rigorous coding process.

Given that the effectiveness of S&T innovation is closely linked to researchers, research institutions and management departments, and the research environment, this study primarily begins with an exploration of researchers’ perceptions. Utilizing GT and qualitative analysis tools (NVivo 11), the study systematically analyzes these critical factors influencing the enhancement of S&T innovation effectiveness, thereby constructing a corresponding theoretical model and testing the theoretical saturation of the model. Figure 2 that illustrates the analytical framework for the critical factors.

Fig. 2
figure 2

Analytical framework for critical factors influencing the enhancement of scientific and technological innovation effectiveness.

Full size image

It should be noted that, as this paper is an exploratory study, its main purpose is to identify the key factors influencing the improvement of scientific and technological innovation efficiency through grounded theory, rather than to verify preset assumptions. Therefore, this paper does not establish explicit research hypotheses in the traditional sense, and its research problem focuses on “how to identify key factors” rather than on testing causal relationships between hypotheses. The research goal and demonstration path of this paper are to apply qualitative analysis methods to systematically identify the key factors affecting the improvement of scientific and technological innovation efficiency from S&T researchers’ perspective and to empirically rank their importance.

The main characteristics of GT at each stage are as follows: (1) Before implementing the research, no research hypothesis is established; categories are directly extracted from the original data. This data can be collected through interviews, surveys, cases studies, and other multi-source data, selected according to the principles of theoretical sampling and typical case methods. (2) During coding, a three-level coding method is adopted. Based on the logic model of “causal conditions, phenomenon analysis, situational conditions, action strategies, result analysis”, the source data is labeled, conceptualized, and hierarchically categorized, ultimately refining core categories and theoretical models. (3) In the theoretical saturation test, relevant data will be retained or added to test whether new concepts emerge.

To ensure the accuracy and reliability of coding in GT, the following measures have been implemented in this paper: First, other researchers in the research team independently conduct coding and then discuss differences to reach a consensus. Second, statistical tools (SPSS) are applied to assess the reliability and validity of the survey results. Third, the NVivo tool is employed to implement the coding process, retain operation traces, generate the coding manual, and clarify concept definitions and logic. Fourth, during coding, the original data is continuously reviewed, initial concepts are revised, and multiple iterations are achieved through the logical thinking of “concept-category relationships” to avoid subjective bias.

Research sample and data acquisition

In early October 2024, the research team engaged in discussions with researchers from institutions such as the University of Chinese Academy of Sciences, Beijing Information Science and Technology University, and Beijing Forestry University regarding a survey on S&T innovation and its influencing factors. A consensus was reached on the topics of the questionnaire and the methods of the survey. By the end of October 2024, the research team developed an anonymous survey questionnaire titled “Critical Factors Influencing the Enhancement of Scientific and Technological Innovation Effectiveness” with an open-ended question: “What critical factors do you believe influence the improvement of researchers’ S&T innovation effectiveness?”.

To ensure the efficiency and rigor of the survey data collection, the study utilized the “Questionnaire Star” survey tool for online distribution and collection, enabling researchers to gather written responses from each participant. In early November 2024, the survey was officially distributed to researchers from universities and research institutions across China. By early December 2024, a total of 296 valid questionnaires were collected. After removing invalid responses, 286 valid questionnaires were confirmed, accounting for 96.73% of the total. The collected text amounted to approximately 12,000 words. Finally, after organizing the data and screening the information, the original survey data were refined to be suitable for GT analysis.

Ethics approval and consent

The conduct of this survey does not require special approval or consent, as stated herein. The survey questionnaire, designed for this study, was distributed through the “Questionnaire Star” platform and is anonymous. The introduction to the questionnaire explicitly indicates that it is an ‘anonymous survey’, ensuring that participation is entirely voluntary. Participants’ privacy and personal data are fully protected, and no personally identifiable information was collected.

Modelling construction and testing

Coding of critical factors and categorization

Open coding: determination of initial categories

Open coding is a process that involves the use of classification, induction, and comparison techniques to analyze and interpret the meanings embedded in the collected information or data35. During the open coding phase, each participant’s textual responses are manually examined, typically at the level of individual words or sentences, to deconstruct and categorize the raw qualitative data. This analytical process leads to the formation of initial categories, where the researcher applies a rational and objective approach to classify, aggregate, and synthesize the raw qualitative data.

In the study, the NVivo11 qualitative coding tool was utilized to analyze 286 open-ended responses provided by researchers in the survey. Each response was scrutinized at the word level to identify key themes and concepts, which were subsequently categorized. Initial categories were generated based on the frequency of concepts appearing at least twice across the responses. Following the normalization and categorization process, a total of 24 initial categories (labeled a01 to a24) were identified, as detailed in Table 1.

Table 1 Open coding of critical factors influencing the enhancement of S&T innovation effectiveness.
Full size table

Spindle type coding: determination of primary and secondary categories

To identify the causal relationships among the initial categories, this study employed spindle type coding. Spindle type coding involves inductively analyzing the logical, structural, and functional attributes of initial categories obtained through open coding36.

In the coding process, the study utilized node clustering analysis results from NVivo 11 and examined the inherent relationships among the meanings of initial categories for cluster integration. Twelve secondary categories were extracted, including research skill levels, research innovation awareness, research intrinsic motivation, research collaboration and support, research assurance support, research policy support, research innovation support, research resource support, research institution management, technological innovation management, research outcome management, and research project management.

Based on the interrelation among the twelve secondary categories, the study integrated their meanings and types. The relationships between the primary categories and secondary categories are as follows:

  1. (1)

    Primary category A: Researcher professional skills, with secondary categories A1 (Research skill levels), A2 (Awareness of research innovation), and A3 (Research intrinsic motivation);

  2. (2)

    Primary category B: Research team collaboration, with secondary category B1 (Research collaboration);

  3. (3)

    Primary category C: Research policy and incentive mechanisms, with secondary category C1 (Research policy and resource support);

  4. (4)

    Primary category D: Construction of a research innovation environment, with secondary categories D1 (Protection of researcher rights), D2 (Research innovation atmosphere and condition), and D3 (Research platform support);

  5. (5)

    Primary category E: Research management and organizational support, with secondary categories E1 (Research institution management), E2 (Scientific and technological innovation management), E3 (Management of research outcomes), and E4 (Research project management).

The specific relationships and connotations of these categories are shown in Table 2.

Table 2 Axical coding of critical factors influencing the enhancement of S&T innovation effectiveness.
Full size table

Selective coding: determination of core categories

Selective coding is grounded in the primary categories derived from spindle type and utilizes deep abstract reasoning to analyze the relationships between core categories and primary categories, thereby extracting the underlying relational structure model behind specific issues or phenomena37. Therefore, the selective coding process necessitates the extraction and description of primary and secondary categories from the “narrative context” of the original qualitative data, as well as the refinement of any potentially overlooked details. This process further clarifies and analyzes the structural relationships between the meanings of the core category and the various hierarchical categories. Through a systematic analysis of the initial qualitative data, this study identifies the core category as “Enhancement of S&T innovation effectiveness” and establishes relational connections between the core category and various hierarchical categories using a “storyline” approach. The GT analytical framework is then constructed as follows:

  1. (1)

    In the realm of S&T innovation, research skills, innovation awareness, and intrinsic motivation collectively form the foundational elements for the enhancement of innovation effectiveness among researchers.

  2. (2)

    A supportive atmosphere for communication and collaboration within research teams provides solid conditions for the positive enhancement of researchers’ S&T innovation effectiveness.

  3. (3)

    Research policy and resource support factors, such as evaluation of research outcomes, performance incentives, resource allocation, compensation and reward systems, are the most direct drivers in ensuring and promoting the continuous enhancement of innovation effectiveness among researchers.

  4. (4)

    Factors related to the S&T innovation environment, including protection of researchers’ rights, innovation atmosphere, academic conditions, and research platform support, are closely linked to the activities of researchers and provide indirect support for further enhancing their research capabilities.

  5. (5)

    Research institutions play a crucial role in directly supporting innovation effectiveness by providing functions such as policy support, institutional management, innovation management, research outcome management, and project management, thereby driving the high-quality enhancement of innovation effectiveness.

Following this process of generation and evolution, with the support and safeguarding of critical factors such as research team collaboration, research policy and incentive mechanism, S&T innovation environment, and research management and organization, researchers’ innovation awareness and intrinsic motivation are bolstered. This, in turn, results in the development of their innovation capabilities and the improvement of their innovation effectiveness.

The findings of this research indicate that the enhancement of S&T innovation and innovation effectiveness is influenced by factors such as researchers’ skills, team collaboration, policies and incentive, innovation environment, and institutional management. Consequently, this study identifies the “Enhancement of S&T Innovation Effectiveness” as the core category, with researchers’ professional skills, team collaboration, policy and incentive, innovation environment, and research management and organization summarized as the main categories.

The construction of a critical factor model

To further understand the relationships between the five core categories associated with the enhancement of S&T innovation effectiveness and the critical factors within each category, this study constructs a conceptual model illustrating the relationships among the core categories, primary categories, and secondary categories, as shown in Fig. 3.

Fig. 3
figure 3

Relationship model of categories for enhancing S&T innovation effectiveness.

Full size image

Theoretical saturation test

A systematic and comprehensive construction of GT requires verification of theoretical saturation to ensure the scientific validity of the research findings. Theoretical saturation refers to the stage at which no new concepts or categories emerge from newly acquired information or data38. To validate this, the study analyzed data from research literature and 10 additional survey questionnaires. No new concepts or categories were identified, and the relationships among the existing categories remained unchanged. This indicates that the constructed category system exhibits good reliability and validity, thereby confirming its theoretical saturation.

Measurement and interpretation of the importance of critical factors

Design and distribution of measurement scales

Using a GT approach, this study developed a category system for the “Enhancement of S&T Innovation Effectiveness”, which encompasses the following five dimensions: “researchers’ professional skills, research team collaboration, construction of the research innovation environment, research management and organizational support, and research policy and incentive mechanism”. Building on this framework, the study incorporates a review of existing literature on the factors influencing S&T innovation effectiveness13,39. Along with a comprehensive analysis of these factors. Additionally, consultations with researchers from institutions such as the University of the Chinese Academy of Sciences and Beijing University of Information Science and Technology were carried out. Based on this foundation, a “Measurement Scale for the Importance of Critical Factors Affecting Scientific and Technological Innovation Effectiveness” was designed and finalized. This scale aims to measure and validate the theoretical model of innovation effectiveness, as well as assess the importance of various critical factors, thereby providing insights for the formulation and improvement of science and technology policies. The measurement scale is shown in Table 3.

Table 3 Importance measurement of critical factors in enhancing S&T innovation effectiveness.
Full size table

The “Measurement Scale for the Importance of Critical Factors Affecting Scientific and Technological Innovation Effectiveness” consists of five items and 35 response options. Specifically, items 6–10 correspond to the following categories: (1) Construction of a research innovation environment category (with 8 options), (2) Research team collaboration category (with 7 options), (3) Research management and organizational support category (with 7 options), (4) Researcher’ professional skills category (with 5 options), and (5) Research policy and incentive mechanisms (with 8 options). The design of these items and options comprehensively reflects and encompasses the 24 initial categories and 12 secondary categories derived from qualitative analysis.

In mid-December 2024, this questionnaire was distributed to universities and research institutions across China. By early January 2025, a total of 306 valid responses had been collected. The data from the questionnaire, and item variables, were subjected to reliability and validity testing using SPSS AU. The results indicated that the KMO (Kaiser–Meyer–Olkin) coefficient for the measurement scale was 0.692, and Bartlett’s test of sphericity showed a significant result at the 0.001 level. Furthermore, the Cronbach’s Alpha values for all five items were greater than 0.70, indicating that the measurement scale demonstrates a satisfactory level of reliability.

Analysis of overall characteristics of the survey data

According to the statistics, the survey sample exhibits an equal distribution of male and female researchers, each comprising 50%. Furthermore, among the surveyed individuals, 189 are aged between 40 and 60 years (representing 61.76%), while 15 are over 60 years old (representing 4.9%). Regarding educational qualifications, 188 individuals hold a doctoral (or postdoctoral) degree (representing 61.49%), and 190 individuals possess an associate senior or senior professional title (representing 62.09%). The surveyed sample shows good consistency in terms of age, education, and professional titles. Additionally, the survey covers 29 provinces and municipalities (including direct-controlled municipalities) in China, providing a broad geographical scope. Among these, 139 responses come from Beijing, the capital, accounting for 45.42%. The geographical distribution of the survey data is presented in Table 4.

Table 4 Distribution of surveyed researchers by province (Top 10).
Full size table

Identification and interpretation of critical factor importance

Identification of the importance of five dimensional critical factors

Through GT and qualitative analysis, this study constructs a five-dimensional model that includes “professional skills of scientific researchers–collaboration of scientific research teams–scientific research management and organization–construction of scientific and technological innovation environment–scientific research policy and incentive mechanism”. There is progressive support and collaboration among these dimensions:

  1. (1)

    The professional skills of scientific researchers serve as the core driving force, directly determining the quality of individual innovation ability and scientific research output, and acting as the foundation for the other dimensions.

  2. (2)

    Scientific research team collaboration enhances the collective application efficiency of professional skills through knowledge sharing and resource integration.

  3. (3)

    Scientific research management and organizational support provide institutional guarantees for team collaboration (such as project management and time allocation) and optimize the fluidity of the innovation process.

  4. (4)

    The construction of the scientific research and innovation environment (such as hardware facilities and academic atmosphere) indirectly influences the innovation drive of individuals.

  5. (5)

    As an external orientation, scientific research policies and incentive mechanisms directly impact the enthusiasm of scientific researchers through resource allocation and performance evaluation.

The second question in the measurement scale evaluates the importance of five dimensions, and calculates the average score for each dimension (i.e., total score ÷ number of valid questionnaires). The average scores are presented in Table 5.

Table 5 Discriminant of the importance of the five dimensional critical factors.
Full size table

Based on the results presented in Table 5, researchers’ professional skills (scoring 2.22) rank the highest, followed by research policy and incentive mechanisms (scoring 2.35) in second place. The factor of research team collaboration (scoring 3.25) ranks in the middle, while the construction of a research innovation environment (scoring 3.57) and research management and organizational support (scoring 3.59) have nearly identical scores and are ranked lowest.

The discriminant analysis results indicate that respondents generally perceive the professional skills of researchers as the most critical factor influencing the enhancement of S&T innovation effectiveness. Therefore, other dimensional factors should concentrate on cultivating and enhancing researchers’ professional skills, providing corresponding innovation support. Furthermore, research management agencies at all levels, along with research institutions, should emphasize the positive role of science and technology policy and incentive mechanisms. These institutions should also foster research teams with strong collaborative awareness and an innovative atmosphere, offering necessary research resources and innovation environments to stimulate researchers’ creativity and initiative.

It can be seen from the above, the essence of the relationships among these dimensions is as follows:

  1. (1)

    The key to professional skills: Professional skills serve as the “endogenous driving force” for improving innovation effectiveness; team collaboration and management serve as the “organizational support”; environmental construction acts as the “incubation condition”; and policy incentives function as the “institutional leverage”. This hierarchical relationship indicates that if professional skills are lacking, the effectiveness of the other dimensions may be compromised.

  2. (2)

    Dominance of internal factors: The core position of scientific researchers is determined by the endogeneity of their professional skills, while the policy incentive mechanism amplifies this internal drive through external incentives (such as salary incentives that directly enhancing research motivation).

  3. (3)

    Nonlinear effect of the external environment: Environmental construction and management support act as “threshold conditions”. Once the basic guarantee level is achieved, further optimization may yield less efficiency improvement than expected.

  4. (4)

    Policy sensitivity and fairness concerns: An unfair distribution of scientific research resources can cause fairness anxiety, leading to fluctuations in the effectiveness of policy incentives.

Selection and explanation of the importance of specific critical factors

  1. (1)

    Researchers’ professional skills dimension: Among all the options, the research initiative and intrinsic motivation for innovation had the highest selection rate at 84.97%. This indicates that respondents generally perceive intrinsic motivation as a crucial factor influencing researchers’ enthusiasm and innovative capabilities in their research activities. Following this, the research knowledge and overall competence (80.39% selection rate) and the ability to apply and implement research skills (74.51% selection rate) also received high selection rates, emphasizing that a solid academic foundation and the capacity to apply skills are essential for enhancing the effectiveness of S&T innovation. Conversely, the research collaborative ability and teamwork awareness (60.78%) and research problem-solving level and influence (58.82%) exhibited relatively lower selection rates, suggesting that there is room for improvement in areas such as team cooperation and personal influence within research contexts.

  2. (2)

    Research team collaboration dimension: The selection data indicate that the investment in research resources (74.51% selection rate) and the research academic atmosphere (74.18% selection rate) are regarded by researchers as the critical factors influencing the enhancement of research effectiveness. Sufficient research resources and a supportive academic environment are essential for fostering researchers’ initiative and innovative capabilities. Additionally, the research innovation incentives (68.3%) and the research team collaboration and coordination (64.71%) also received high selection rates, suggesting that the level of collaboration and innovation incentives among research teams significantly contribute to improving S&T innovation effectiveness. In contrast, the sense of research mission and responsibility (38.89%) and the resource sharing within the research team (40.2%) are of relatively less importance, indicating that these factors have a more limited impact on improving S&T innovation effectiveness.

  3. (3)

    Research management and organizational support dimension: According to the survey data, the rationality of the research evaluation system (83.33% selection rate) ranks first, highlighting its significant influence. Following this, the adequacy of research time (66.99%) and the awareness and level of research innovation (66.01%) are also critical factors that affect the enthusiasm and innovative capacity of researchers. Furthermore, the rationality of research team construction (58.5%) demonstrates its substantial impact on improving research effectiveness. In comparison to other factors, the research logistical support (46.41%) and the availability of academic resources and training levels (53.59%) are selected less frequently, suggesting that their impact is relatively weaker. Lastly, the research implementation and breakthrough capabilities (33.66%) received the lowest selection rate, indicating a limited role in enhancing research outcomes.

  4. (4)

    Construction of a research innovation environment dimension: Regarding the importance of critical factors, research performance and salary incentives (80.72% selection rate) and research culture and academic atmosphere (74.18% selection rate) are the most frequently selected. These factors are considered essential for boosting researchers’ enthusiasm for S&T innovation and enhancing their innovative abilities. Furthermore, the research policy and institutional framework (67.32%) also show considerable influence. In contrast, the impact of research hardware and software configuration (54.25%) and the organization and allocation of research resources (50.98%) is comparatively less pronounced. Additionally, the research support and backing (47.39%) and the research skills and overall competence (38.89%) are the least selected, indicating that these factors have a relatively limited effect on researchers and the effectiveness of S&T innovation. Lastly, the research team collaboration and coordination (27.45%) is the least influential factor.

  5. (5)

    Research policy and incentive mechanisms: The data analysis indicates that the difficulty and fairness of research project applications (78.1%) and the rationality of project funding and resource allocation (74.51%) received the highest selection rates. This suggests that researchers are particularly concerned with these two factors, emphasizing the critical importance of equal access to research opportunities and rational resource allocation in enhancing their initiative and the overall effectiveness of S&T innovation. Furthermore, the shifts in research direction and funding allocation (55.23%) also garnered a relatively high selection rate, reflecting researchers’ strong expectations for adaptability to policy changes and stable financial support. Additionally, the rationality of research policy formulation (46.73%) and the efficiency of research management processes (44.77%) were similarly selected, indicating a clear demand for rational policies and efficient management processes. In contrast, factors such as the strength of intellectual property protection (36.27%), the industry-academia-research collaboration (34.97%), and the level of research organization and management (39.22%) received comparatively lower selection rates, suggesting that these factors currently have a limited impact on enhancing the proactivity and effectiveness of S&T innovation at present.

Discussion

Analysis of the critical factors impacting S&T innovation effectiveness

Based on the results from the critical factor importance measurement scale survey, significant differences are observed in the factors influencing the effectiveness of S&T innovation across various dimensions. These differences reflect the diverse needs and concerns of researchers regarding research activities. Below is a brief analysis of the key factors within key dimension:

Internal driving force: plays a leading role in shaping innovation effectiveness for external resources

The survey highlights research innovation proactivity and intrinsic motivation as the most prioritized secondary factor (84.97% selection rate), surpassing tangible variables like funding allocation and platform support. This aligns with behavioral theories positing that self-determination and intellectual curiosity act as catalytic forces for sustained innovation output40,41, a phenomenon particularly evident in China’s high-pressure academic environment. Paradoxically, while policy discussions often emphasize resource inputs (e.g., R&D expenditure or hardware infrastructure)42, the data suggest that institutional mechanisms to foster intrinsic motivation—such as flexible evaluation systems and autonomy in research agendas—remain underdeveloped. This misalignment implies that current innovation policies may overemphasize extrinsic incentives (e.g., financial rewards) while neglecting psychological ownership and intellectual engagement, thereby limiting the sustainability of innovation efforts.

Policy design nuances: equity and stability should be regarded as important balance levers

Despite ranking second in overall importance, the fairness of project application processes (78.10%) and the rationality of resource distribution (74.51%) emerged as critical pain points, reflecting systemic inequities in China’s innovation ecosystem. Comparative analysis with regional disparity studies reveals a geographical dimension to these concerns: resource concentration in eastern provinces intensifies perceived unfairness among researchers in less developed regions. However, the study’s data also indicate that transparency in policy execution—rather than funding volume alone—determines trust in institutional frameworks43. This underscores the necessity of decentralizing funding decisions and establishing cross-regional collaboration platforms to mitigate Matthew effects, a challenge scarcely addressed by current national S&T major projects44.

The paradox of team collaboration: being paid attention to in appearance yet underperforming in practice

While team collaboration ranked third among overall determinants (64.71% selection rate), qualitative feedback from open-ended survey responses reveals a disconnect between theoretical synergy and operational realities. Respondents frequently cited “administrative interference in team formation” and “lack of cross-disciplinary recognition mechanisms” as invisible barriers—issues absent from conventional metrics like co-authorship counts. This echoes prior criticisms of China’s innovation system favoring hierarchical, discipline-siloed structures over organic knowledge networks. To resolve this paradox, policy interventions must shift from promoting collaboration as a numerical KPI (e.g., joint patents) to fostering institutionalized platforms for intellectual risk-sharing mechanism.

These insights call for a reimagining of China’s technology innovation governance: moving from input-driven quantitative expansion to cultivating organic ecosystems where intrinsic motivation, spatial equity, and adaptive collaboration converge as mutually reinforcing pillars. Future research could longitudinally track how evolving incentive structures (e.g., blended metrics balancing individual merit and team contributions) influence these dynamics.

Theoretical and practical implications

This study offers both theoretical and practical contributions to the field of S&T innovation. Firstly, the research illustrates the value of the “qualitative analysis + scale measurement” approach, a research paradigm that effectively merges theory and practice. This approach has proven to be a powerful tool for analyzing complex issues and their solutions, presenting significant potential for advancing research in the domain of scientific and technological innovation.

Secondly, the study employed qualitative analysis of data regarding researchers’ perceptions of innovation effectiveness, leading to the identification of five primary categories—“Researcher professional skills, Research team collaboration, Research management and organizational support, Construction of a research innovation environment, and Research policy and incentive mechanisms”—derived from 24 initial categories.

Finally, the research uncovered 12 secondary categories, including factors such as research skills level, ultimately culminating in the core category of “Enhancement of S&T innovation effectiveness”. This resulted in the creation of a systematic theoretical framework for the key influencing factors, providing a robust foundation for future research and offering essential indicators for policy formulation. Moreover, through the application of a critical factor measurement scale, the study identified the key dimensions within the five primary categories and ranked the importance of the critical factors within each dimension, thereby emphasizing the priority areas for improvement. It suggests that research management institutions and scientific organizations should prioritize enhancing researchers’ professional skills when developing and refining scientific policies, offering valuable empirical support for policy formulation and implementation.

In summary, this study enhances both the theoretical understanding of the factors influencing S&T innovation effectiveness and provides practical guidance for developing policies designed to promote innovation within the scientific community.

Conclusions and recommendations

Conclusions

This study employs the GT methodology, based on survey data regarding researchers’ perceptions of the enhancement of S&T innovation effectiveness. Utilizing qualitative analysis tools, a categorical system and theoretical model of critical factors were developed. Additionally, a questionnaire survey and measurement scale were employed to assess and empirically compare the significance of the critical factors influencing S&T innovation effectiveness. The research findings are as follows:

First, based on the initial category results obtained through open coding, it was found that the areas receiving the most attention in the statistics were research output evaluation and performance incentives, investment and allocation of research resources, and the innovation environment and academic atmosphere. This indicates that researchers place significant emphasis on the guidance and scientific nature of research performance evaluation and incentives, as well as the fairness of research resource allocation. Furthermore, researchers are more inclined to engage in research activities within a favorable innovation environment and academic atmosphere. Investment and allocation of research resources fall under the ‘hard environment,’ while output evaluation, performance incentives, innovation environment, and academic atmosphere belong to the ‘soft environment.’ Therefore, policies that integrate both ‘hard’ and ‘soft’ elements are essential for effectively motivating and encouraging S&T innovation among researchers.

Second, the importance ranking results of the five critical factors indicate that the enhancement of S&T innovation effectiveness is influenced by multiple elements, each exerting a different impact. The highest-ranking factor was the professional skills of researchers, followed by research policy and incentive mechanisms, which ranked second and third, respectively, along with research team collaboration. The research innovation environment, and research management and organizational support ranked fourth and fifth. This suggests that researchers generally believe that the key foundation for improving research innovation effectiveness is the enhancement of their professional skills, while ensuring the sustainability of innovation effectiveness relies on robust research policies and incentive mechanisms. This conclusion aligns with the first finding.

Policy recommendations for enhancing the S&T innovation effectiveness

The study finds that researchers’ professional skills are the most critical factor in enhancing the effectiveness of S&T innovation. Other dimensions should focus on fostering and improving researchers’ professional skills, providing the essential support for S&T innovation. Furthermore, research management institutions and organizations at all levels should recognize the positive role of science and technology policies and incentive mechanisms, and strive to establish collaborative, synergistic research teams by providing necessary research resources and an innovation-friendly environment. This will stimulate the creative potential and initiative of researchers.

Consequently, the study also suggests that the organic collaboration of four mechanisms can be achieved through hierarchical talent development, collaborative team building, unimpeded dual-line communication, and the optimization of science and technology policies and incentives. The specific recommendations are as follows:

Establishing a hierarchical talent development mechanism

  1. (1)

    Categorized training with clear development paths: Develop structured training plans tailored to the career stages of researchers (e.g., early-career scholars, mid-level researchers, senior scientists), with specific objectives for professional training and research capabilities.

  2. (2)

    Mentorship system: Engage senior experts or industry leaders as mentors, providing technical guidance, sharing project experiences, and offering career development advice. Cultivate a “mentor–mentee” relationship for early-career researchers, focusing on both practical and innovative skills.

  3. (3)

    Multi-channel practical training: Facilitate domestic and international academic exchanges through short visits and collaborative research to expand research perspectives. Encourage researchers to engage in industry-specific technology development or project collaborations to strengthen their skills in research application and technology transfer.

  4. (4)

    Regular skill assessment: Implement regular skill assessments and performance reviews to ensure efficient allocation of training resources and allow for adjustments to training plans based on individual performance.

Strengthening collaborative mechanisms within research teams

  1. (1)

    Optimizing team structure and role distribution: Emphasize interdisciplinary collaboration and professional complementarity, ensuring a balanced and rational allocation of expertise and responsibilities within the team to prevent resource duplication or conflicts.

  2. (2)

    Establishing regular communication protocols: Encourage regular internal academic seminars for sharing research progress and exchanging innovative ideas. Enhance information sharing among team members through digital collaboration platforms (e.g., shared documents or collaborative software) to boost communication and efficiency.

  3. (3)

    Stimulating team innovation: Recommend the implementation of flexible incentive policies that link team research outcomes to individual contributions. Foster interdisciplinary collaboration and promote deep integration across various disciplines to ignite potential innovative opportunities.

  4. (4)

    Effective conflict resolution mechanisms: Develop management protocols for team construction and collaboration, providing clear procedures for resolving disagreements and conflicts. Engage independent third parties (e.g., research managers) to facilitate internal coordination.

Facilitating dual-line communication in research

  1. (1)

    Establishing regular communication channels: Set fixed “Researcher-Manager Communication Days” for consistent face-to-face discussions on project progress, policy adjustments, and more. Introduce digital communication platforms, such as online feedback channels, to ensure prompt responses to researchers’ suggestions.

  2. (2)

    Implementing bidirectional feedback mechanisms: Ensure that management personnel provide timely explanations of policy changes to maintain transparency, while researchers offer constructive feedback on policy and management process improvements.

  3. (3)

    Creating dedicated liaison roles: Establish roles like “Research Affairs Coordinators” to focus on policy interpretation, management implementation, and gathering researchers’ needs, thereby enhancing communication efficiency.

  4. (4)

    Strengthening cross-departmental collaboration: Encourage collaboration among research management, administrative management, and technical support departments to create an integrated support network, reducing communication barriers.

Improving science and technology policy and incentive mechanisms

  1. (1)

    Multilevel reward system: Establish incentive policies aimed at various stages of researchers’ careers, such as innovation awards for early-career researchers and collaboration rewards for interdisciplinary projects. Tailor incentive policies to three research domains: basic research, applied research, and outcome transformation.

  2. (2)

    Improving fund allocation precision: Enhance surveys on the needs of researchers and teams, prioritizing resources for high-potential and high-impact projects. Introduce dynamic funding mechanisms that adjust financial support based on the project’s stage and results.

  3. (3)

    Enhancing technology transfer support: Create a robust system for transferring research results, introduce a “Research-Industry Connection Fund”, and encourage research projects that align with market demands. Promote the establishment of technology testing bases and incubators to facilitate the industrial application of research innovations.

  4. (4)

    Strengthening policy implementation: Conduct regular evaluations of existing policies’ effectiveness and make timely adjustments as necessary. Promote policy interpretation training for researchers to ensure they fully understand and can effectively utilize available policy resources.

Outlook

However, there are several limitations in this study: First, the data categorization and inductive processes in grounded theory are inherently subjective, necessitating further validation for the coding results and the determination of model variables. Second, the sample size and scope of the survey and scale measurements are relatively limited, and the professionalism and cognitive depth of the survey participants require further enhancement. Given that the effectiveness of S&T innovation is influenced by multiple factors and exhibits nonlinear characteristics, future research could integrate system dynamics and simulation tools to explore and analyze the nonlinear relationships between key influencing factors. Additionally, simulation analysis of the effects of science and technology policies could provide valuable insights for research management institutions and scientific organizations, offering guidance and potential pathways for advancing reforms in the science and technology system.