This book presents contemporary empirical methods in software engineering related to the plurality of research methodologies, human factors, data collection and processing, aggregation and synthesis of evidence, and impact of software engineering research. The individual chapters discuss methods that impact the current evolution of empirical software engineering and form the backbone of future research. Following an introductory chapter that outlines the background of and developments in empirical software engineering over the last 50 years and provides an overview of the subsequent contributions, the remainder of the book is divided into four parts: Study Strategies (including e.g. guidelines for surveys or design science); Data Collection, Production, and Analysis (highlighting approaches from e.g. data science, biometric measurement, and simulation-based studies); Knowledge Acquisition and Aggregation (highlighting literature research, threats to validity, and evidence aggregation); and Knowledge Transfer (discussing open science and knowledge transfer with industry). Empirical methods like experimentation have become a powerful means of advancing the field of software engineering by providing scientific evidence on software development, operation, and maintenance, but also by supporting practitioners in their decision-making and learning processes. Thus the book is equally suitable for academics aiming to expand the field and for industrial researchers and practitioners looking for novel ways to check the validity of their assumptions and experiences. Chapter 17 is available open access under a Creative Commons Attribution 4.0 International License via link.springer.com.

This book gathers chapters from some of the top international empirical software engineering researchers focusing on the practical knowledge necessary for conducting, reporting and using empirical methods in software engineering. Topics and features include guidance on how to design, conduct and report empirical studies. The volume also provides information across a range of techniques, methods and qualitative and quantitative issues to help build a toolkit applicable to the diverse software development contexts

Like other sciences and engineering disciplines, software engineering requires a cycle of model building, experimentation, and learning. Experiments are valuable tools for all software engineers who are involved in evaluating and choosing between different methods, techniques, languages and tools. The purpose of Experimentation in Software Engineering is to introduce students, teachers, researchers, and practitioners to empirical studies in software engineering, using controlled experiments. The introduction to experimentation is provided through a process perspective, and the focus is on the steps that we have to go through to perform an experiment. The book is divided into three parts. The first part provides a background of theories and methods used in experimentation. Part II then devotes one chapter to each of the five experiment steps: scoping, planning, execution, analysis, and result presentation. Part III completes the presentation with two examples. Assignments and statistical material are provided in appendixes. Overall the book provides indispensable information regarding empirical studies in particular for experiments, but also for case studies, systematic literature reviews, and surveys. It is a revision of the authors’ book, which was published in 2000. In addition, substantial new material, e.g. concerning systematic literature reviews and case study research, is introduced. The book is self-contained and it is suitable as a course book in undergraduate or graduate studies where the need for empirical studies in software engineering is stressed. Exercises and assignments are included to combine the more theoretical material with practical aspects. Researchers will also benefit from the book, learning more about how to conduct empirical studies, and likewise practitioners may use it as a “cookbook” when evaluating new methods or techniques before implementing them in their organization.

Nowadays, societies crucially depend on high-quality software for a large part of their functionalities and activities. Therefore, software professionals, researchers, managers, and practitioners alike have to competently decide what software technologies and products to choose for which purpose. For various reasons, systematic empirical studies employing strictly scientific methods are hardly practiced in software engineering. Thus there is an unquestioned need for developing improved and better-qualified empirical methods, for their application in practice and for dissemination of the results. This book describes different kinds of empirical studies and methods for performing such studies, e.g., for planning, performing, analyzing, and reporting such studies. Actual studies are presented in detail in various chapters dealing with inspections, testing, object-oriented techniques, and component-based software engineering.

This book provides guidelines for practicing design science in the fields of information systems and software engineering research. A design process usually iterates over two activities: first designing an artifact that improves something for stakeholders and subsequently empirically investigating the performance of that artifact in its context. This “validation in context” is a key feature of the book - since an artifact is designed for a context, it should also be validated in this context. The book is divided into five parts. Part I discusses the fundamental nature of design science and its artifacts, as well as related design research questions and goals. Part II deals with the design cycle, i.e. the creation, design and validation of artifacts based on requirements and stakeholder goals. To elaborate this further, Part III presents the role of conceptual frameworks and theories in design science. Part IV continues with the empirical cycle to investigate artifacts in context, and presents the different elements of research problem analysis, research setup and data analysis. Finally, Part V deals with the practical application of the empirical cycle by presenting in detail various research methods, including observational case studies, case-based and sample-based experiments and technical action research. These main sections are complemented by two generic checklists, one for the design cycle and one for the empirical cycle. The book is written for students as well as academic and industrial researchers in software engineering or information systems. It provides guidelines on how to effectively structure research goals, how to analyze research problems concerning design goals and knowledge questions, how to validate artifact designs and how to empirically investigate artifacts in context – and finally how to present the results of the design cycle as a whole.

This book addresses action research (AR), one of the main research methodologies used for academia-industry research collaborations. It elaborates on how to find the right research activities and how to distinguish them from non-significant ones. Further, it details how to glean lessons from the research results, no matter whether they are positive or negative. Lastly, it shows how companies can evolve and build talents while expanding their product portfolio. The book’s structure is based on that of AR projects; it sequentially covers and discusses each phase of the project. Each chapter shares new insights into AR and provides the reader with a better understanding of how to apply it. In addition, each chapter includes a number of practical use cases or examples. Taken together, the chapters cover the entire software lifecycle: from problem diagnosis to project (or action) planning and execution, to documenting and disseminating results, including validity assessments for AR studies. The goal of this book is to help everyone interested in industry-academia collaborations to conduct joint research. It is for students of software engineering who need to learn about how to set up an evaluation, how to run a project, and how to document the results. It is for all academics who aren’t afraid to step out of their comfort zone and enter industry. It is for industrial researchers who know that they want to do more than just develop software blindly. And finally, it is for stakeholders who want to learn how to manage industrial research projects and how to set up guidelines for their own role and expectations.

Empirical verification of knowledge is one of the foundations for developing any discipline. As far as software construction is concerned, the empirically verified knowledge is not only sparse but also not very widely disseminated among developers and researchers. This book aims to spread the idea of the importance of empirical knowledge in software development from a highly practical viewpoint. It has two goals: (1) Define the body of empirically validated knowledge in software development so as to advise practitioners on what methods or techniques have been empirically analysed and what the results were; (2) as empirical tests have traditionally been carried out by universities or research centres, propose techniques applicable by industry to check on the software development technologies they use.