Empirical Software Engineering

Empirical Software Engineering (ESE) is the systematic study of software development, maintenance and evolution through observation, measurement and experimentation. Departing from purely theoretical or ad hoc practices, ESE draws on surveys, case studies, controlled and quasi‐experiments to generate evidence about processes, tools and techniques. Core goals include characterising real‐world phenomena, predicting outcomes, evaluating innovations and guiding decision‐making. Rigorous attention to validity—internal, external, construct and conclusion—ensures that findings reflect causal relationships rather than artefacts of context or measurement. Mixed‐method designs blend quantitative metrics with qualitative insights, while replication studies confirm or refute prior results across domains and scales. Mining software repositories, statistical modelling and controlled comparisons all contribute to a growing body of scientific knowledge. By translating empirical insights into practice, ESE accelerates the adoption of effective methods, illuminates the impact of human factors and supports evidence‐based improvements in global software supply chains, critical infrastructures and consumer applications.

Research from Nature Portfolio

Researchers have proposed a low‐cost communication framework for eliciting requirements across globally distributed teams, accounting explicitly for time‐zone differences, cultural norms and language barriers to improve coordination and quality. Advances in handling class‐imbalance in code‐smell severity detection employ principal‐component‐based feature selection combined with synthetic oversampling and multiple machine‐learning classifiers, achieving near‐perfect accuracy in prioritising refactoring efforts. A novel machine‐learning tool automates vulnerability detection in C/C++ codebases of Internet of Things operating systems by training on labeled datasets from open‐source releases and standardized weakness taxonomies, yielding high F-measure scores and outperforming static‐analysis tools. Additionally, a multi‐criteria decision‐analysis approach optimises software‐maintainability models via genetic‐algorithm‐tuned ensembles and TOPSIS ranking to remove uncertainty in predictive metrics for heterogeneous codebases.

Research from all publishers

Recent methodological work has introduced a validated framework for assessing both the methodological rigor and industrial relevance of technology evaluations, helping bridge the gap between academic studies and real‐world practice. Comprehensive guidelines for selecting and applying statistical tests in algorithmic and optimisation research now enable more defensible conclusions about treatment effects and performance claims. A decision‐support structure for designing empirical studies offers researchers a stepwise approach to choosing between surveys, experiments, case studies or mixed methods based on objectives, context and resource constraints. Together, these contributions strengthen the design, analysis and interpretability of evidence in contemporary software engineering research.

Empirical Software Engineering publication trend

The graph below shows the total number of articles in empirical software engineering across all publications each year (not limited to Nature Index journals).

Technical terms

Empirical Software Engineering: The discipline of studying software development phenomena through systematic observation, measurement and experimentation.

Controlled experiment: A study in which one variable (the treatment) is manipulated under randomized or balanced conditions to observe causal effects on outcomes.

Quasi‐experiment: An empirical study like a controlled experiment but without full randomisation, relying on natural groupings or contextual assignments.

Case study: An in‐depth investigation of one or a small number of software projects or processes, using multiple data sources to explore complex phenomena.

Survey: A method for collecting self‐reported data from a sample of practitioners or projects to characterise knowledge, attitudes and practices.

Mixed‐method research: A design that integrates qualitative and quantitative approaches to provide complementary insights and strengthen findings.

Replication study: An empirical investigation that reproduces or extends a previous study to confirm or challenge its results across different settings.

Threat to validity: A factor that may jeopardise the credibility of study findings, classified into internal, external, construct or conclusion validity.

References

1. A cost effective communication model for requirements elicitation in global software development. Scientific Reports (2023).
2. A study of dealing class imbalance problem with machine learning methods for code smell severity detection using PCA-based feature selection technique. Scientific Reports (2023).
3. iDetect for vulnerability detection in internet of things operating systems using machine learning. Scientific Reports (2022).
4. A multiple criteria decision analysis based approach to remove uncertainty in SMP models. Scientific Reports (2022).
5. Empirical Software Engineering.