Learn what really matters based on evidence: An Introduction to Evidence-based Management

What is Science-based and Evidence-based Management?

To be an evidence-based manager is to rely on properly analyzed data and critical, scientific thinking to make substantive decisions. This can be contrasted with making decisions based on anecdotal (story-based) evidence, personal experience, popular trends, or informal theories that have not been adequately tested (Pfeffer & Sutton, 2006). Evidence-based managers conduct reviews of the empirical literature prior to selecting strategies, and root their choices in what the data generally shows, rather than what their gut instincts or personal biases tell them (Bazerman, 2009).

The benefits of taking an evidence-based approach are numerous. Evidence-based management tends to be rooted in the documented, carefully analyzed experiences of many managers, as well as social scientists. In empirical studies, participants groups are large, and include employees at a variety of organizations. This large and diverse sampling increases the accuracy of findings. Conclusions arrived at via evidence-based approaches are less likely to be random, and are more likely to be reproducible and applicable to a wide variety of settings, compared to conclusions arrived at via less scientific means (Sanders, van Riemsdijk, & Groen, 2008).

Evidence-based management also tends to rely on more objective and systematic analyses of data. Human bias can lead to inaccurate conclusions and biased interpretations of events. Managers may, for example, misremember their attempts at introducing a new strategy, recalling only the instances where that strategy worked, and conveniently forgetting or discounting experiences when it did not (Tschan et al, 2009). This can lead to inaccurate conclusions about a strategy’s efficacy. By using the scientific method to more objectively test predictions and analyze data, evidence-based conclusions are higher in accuracy and less likely to be biased.

Defining and locating evidence-based data and information

Individuals without formal scientific training are often unaware of how to seek out trustworthy, high-quality evidence-based information. Outside of social science graduate programs, very few people are trained in how to seek out research and evaluate it. The core principles of finding good evidence, however, are relatively straightforward:

1. Information should come from a peer-reviewed or otherwise appropriately vetted source,
2. methods and data should be fully and clearly documented within that source, and
3. conclusions should be rooted in a grounded, accurate interpretation of results.

These requirements, and how to search for quality evidence-based information, are described in further detail below.

Peer review

High-quality research must have undergone peer review, a lengthy, formalized process wherein at least three experts in the field provide detailed feedback on a report (Spier, 2002). For academic journals, the peer review process is highly selective, with only a small minority of articles ultimately earning publication, following rounds of revisions. The selectivity and expert vetting inherent to this process helps ensure that published work is credible and properly conducted (Jefferson, Wager, & Davidoff, 2002). Peer-reviewed research is most commonly found in academic journals or periodicals, but peer-review is sometimes also performed on chapters of academic texts or on presentations for professional conferences.

Documentation of data and methods

In addition to being peer-reviewed, quality evidence is well documented. Researchers should be willing and able to share their original data with colleagues when requested, along with the results of statistical analyses. This establishes credibility and allows for double-checking of results. In some instances, the full data that an organization or research team has collected may be freely available online. This is particularly the case for national datasets collected by governmental and public agencies. High-quality research should also clearly document its methodology. In a well-written and trustworthy research report, the specifics of how the study was conducted, who participated in the study, and when and where the study took place will be evident to the reader. The instruments that used to measure variables should also be explained, as well as how data was stored. Clarity about methods allows other researchers to reproduce results, which provides additional verification.

Appropriately grounded conclusions

Evidence-based information and conclusions must be firmly grounded in reality, with clear acknowledgement of limitations (Bem, 2004). No study can provide conclusive evidence that a management strategy always works, or that certain trends will always be evident. All research conclusions are specific to the time and place in which the study was conducted, and may be influenced by particular details of the sample demographics, study site, or sample size. Furthermore, all scientific conclusions are tentative, and open to further revision as additional evidence is collected over time. A responsible researcher will acknowledge these and other limitations, and not overstate confidence in their results.

Where to find evidence-based management knowledge?

To find evidence-based management information, seek out research databases such as PsychInfo and JSTOR. These sites compile all the published research from peer-reviewed academic journals, dissertations, and book chapters, and provide digital copies that go back decades. Useful, management-related research can be found in a variety of types of social science, communications, and management journals. Google Scholar is also a popular resource for scouring multiple research databases at a time, and allows for more flexible and dynamic search options. Restricting your search to specific locations or dates (using a database’s advanced search options) can help a manager locate findings relevant to their situation and purpose.

Another source of evidence-based management knowledge you should consider are online learning platforms like CQ. They bridge the gap between social sciences and practice in management by translating years of social science research into distilled, practical advice that is both empirically supported and easy to comprehend. This makes it easier for management practitioners to find, evaluate, interpret and implement high quality management knowledge in their organization without going through a vast amount of complex scientific studies.

Reading and making use of evidence-based information

When looking at peer-reviewed research articles, begin by closely reading the abstract. This brief summary of the study should outline the methodology and the results in fairly understandable language. If the abstract suggests that the article may be relevant to the manager’s questions and goals, the full article should be read (Bem, 2004).

Introduction section: Research question and topic background

The introduction section provides a background to the research question and topic at hand, with numerous citations to existing research. These citations can be located and read as well, for additional information on the topic. You will also find a link to models and theories the research paper relies on in the introduction section. For instance, goal setting theory is a theoretical framework which is the point of departure for many research articles and thus will be briefly introduced in most of those articles.

Methods section: Research approach and implementation

The methods section should describe exactly how the study was conducted, and should further help the reader to determine if the research is relevant to their goals. Depending on the type of the research article the methods section varies. On a high level you can distinguish between qualitative and quantitative research studies. Qualitative research usually relies on methods such as case studies, focus groups, semi-structured interviews or text analysis. Quantitative research is more inclined towards statistics with close-ended surveys, statistical tests and meta-analysis as methods of choice.

Results section: Research outcomes

The results section of a journal article typically features statistical results, which may be daunting to read for managers who lack statistical expertise. However, this section shouldn’t be skipped. Read our introduction to statistics for management practitioners for a primer on the key statistical principles you need to know. When you read the results section focus on the following conceptual points:

• What questions are being tested?
• Do the results support the researcher’s predictions?

This information should be available in the results, and should not require statistical acumen to follow. 

Discussion and conclusion section: Result interpretation and recommendations

Finally, the conclusion and discussion section of the paper should provide a more thorough interpretation of the results, as well as recommendations for future researchers and practitioners.

As you consume evidence-based research, look at the overall trends found across multiple studies, rather than focusing on the isolated findings of a single person. Work to also understand the theories that guide the scientific work.

• Why, according to the author, does an observed phenomenon occur?
• If a particular intervention works, how or why does it work?
• How strong is the presented evidence considering the applied methodology (e.g. single case study versus meta-analysis)?

The larger, theoretical picture is just as vital to understanding the literature as the individual findings are. An awareness of study limitations and flaws is also essential.

Limitations of scientific evidence

Science is an important source of knowledge and has strongly contributed to professionalize management and make work better. However, there are still many areas in management which lack a solid body of evidence as they have been newly discovered or empirical studies have not been conducted yet. In addition, the quality of the available scientific evidence also varies.

Empirical studies with small sample sizes lack the statistical power to draw reliable conclusions and can even be misleading. Academic publishing is not immune to biases as recent discussions about the shortcomings of traditional peer review processes and publication bias show (Dwan et al, 2013). How can you as a management practitioner deal with those flaws imminent in scientific evidence?

The four sources of evidence

You can overcome these limitations of scientific evidence by considering additional sources of knowledge and select the best one for your specific case. In evidence-based management you should consider the following four sources of evidence whereas scientific research is one of them:

1. Organizational data
2. Professional expertise
3. Stakeholder values and concerns
4. Insights from scientific research

When you consider those four sources of evidence and select the best one you will improve your decision-making quality which will add up to an increased organizational performance over time.

References and further readings

Barends, E., Rousseau, D.M., & Briner, R.B. (2014). Evidence-Based Management: The Basic Principles. Amsterdam: Center for Evidence-Based Management.

Bazerman, M.H. (2009). Judgment in Managerial Decision Making. Wiley, New York.

Dwan, K.M., Gamble, C., Williamson, P.R., & Kirkham, J. (2013). Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review. PloS one. https://doi.org/10.1371/journal.pone.0066844

Jefferson, T., Wager, E., & Davidoff, F. (2002). Measuring the quality of editorial peer review. Jama, 287(21), 2786-2790.

Pfeffer, J., & Sutton, R. I. (2006). Evidence-based management. Harvard Business Review, 84(1), 62.

Rousseau, D. M. (2006). Is there such a thing as “evidence-based management”?. Academy of management review, 31(2), 256-269.

Rynes, S.L., Colbert, A.E., Brown, K.G. (2002). HR Professionals' beliefs about effective human resource practices: correspondence between research and practice. Human Resource Management, 41 (2), 149-174.

Sanders, K., van Riemsdijk, M., & Groen, B. (2008). The gap between research and practice: a replication study on the HR professionals’ beliefs about effective human resource practices. The International Journal of Human Resource Management, 19(10), 1976-1988.

Spier, R. (2002). The history of the peer-review process. Trends in Biotechnology, 20 (8), 357-358, ISSN.

Tschan, F., Semmer, N. K., Gurtner, A., Bizzari, L., Spychiger, M., Breuer, M., & Marsch, S. U. (2009). Explicit reasoning, confirmation bias, and illusory transactive memory: A simulation study of group medical decision making. Small Group Research, 40(3), 271-300.