External Validity, Coverage Error and Non-response Bias

From Information Systems at Pitt Business

Like many other mature fields, the IS field suffers from inconsistent empirical findings on essentially the same question. Thus, rather than continuously adding new experiments to the already-existing pile of research, to move a specific research topic forward, it is important to assimilate and integrate previous findings. Numerous review methodologies exist, one of which is meta-analysis.

This positivist method, dominant in many fields in the social and behavioral sciences, synthesizes the quantitative results of many individual empirical studies. As evidenced by its focus on data, it is much less subjective than the other literature review methods and is the only review method that utilizes effect sizes, or the magnitude of the effects between the variables studied. Consequently, studies showing insignificant results can accurately be integrated with studies showing significant results to form a more credible conclusion.

[edit] Abstract

Four methods for reviewing a body of research literature – narrative review, descriptive review, vote-counting, and meta-analysis – are compared. Meta-analysis as a formalized, systematic review method is discussed in detail in terms of its history, current status, advantages, common analytic methods, and recent developments.

Meta-analysis is found to be underutilized in IS. Suggestions on encouraging the use of meta- analysis in IS research and procedures recommended for meta-analysis are also provided.

[edit] Paper Information

Authors: William King, Jun He

To read more check out the paper at External Validity, Coverage and Nonresponse Errors in IS Survey Research

This article was originally published in CAIS, Vol 16, 2005.

[edit] Keywords

Literature reviews, narrative review, descriptive review, vote counting, meta-analysis