Description
This course in the interdisciplinary Ph.D. in evaluation program at Western Michigan University is an advanced graduate seminar designed to provide students with the knowledge, skills and abilities necessary to conduct basic research reviews, research syntheses and meta-analyses. Topics covered include, but are not limited to:
- The increasing use of meta-analysis in formulating and enacting evidence-based policies and practices.
- The role of meta-analysis in theory development.
- Principles and procedures for planning and executing research reviews and meta-analyses
- Identifying and retrieving literature.
- Coding studies.
- Computing effect sizes (e.g., based on means, binary data, and correlations) and their corresponding confidence intervals for meta-analysis.
- Converting among effect sizes.
- Factors that affect precision (e.g., variance, standard error, confidence intervals).
- Fixed-effect and random-effects models for meta-analysis.
- Identifying and quantifying heterogeneity.
- Prediction intervals.
- Subgroup analysis.
- Meta-regression.
- Meta-analysis with complex data structures.
- Power analysis for meta-analysis.
- Publication bias.
- Psychometric meta-analysis.
Students should have at least a fundamental knowledge of applied statistics and experimental and quasi-experimental design to succeed in the course and will be required to plan and execute a basic meta-analysis. EMR 6550: Experimental and Quasi-Experimental Designs, is a recommended, but not required, prerequisite.
Syllabus
Instructor
Required textbooks
- Bornenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. West Sussex, UK: Wiley.
- Cooper, H., Hedges, L. V., & Valentine, J. C. (Eds.). (2009). The handbook of research synthesis and meta-analysis (2nded.). New York, NY: Russell Sage Foundation.
- Comprehensive Meta-Analysis 2.0
Required readings
These readings are for instructional purposes only.
- Anderson, C. A., et al. (2010). Violent video game effects on aggression, empathy, and prosocial behavior in eastern and western countries: A meta-analytic review. Psychological Bulletin, 136(2), 151-173.
- Beretavs, S. N. (2010). Meta-analysis. In G. R. Hancock, G. R. R. O. & Mueller (Eds.), The reviewer’s guide to quantitative methods in the social sciences (pp. 255-264). New York, NY: Routledge.
- Bornmann, L., diger Mutz, R., & Daniel, H-D. (2010). A reliability-generalization study of journal peer reviews: A multilevel meta-analysis of inter-rater reliability and its determinants. PLoS ONE, 5(2), e14331.
- Cook, T. D., & Leviton, L. C. (1980). Reviewing the literature: A comparison of traditional methods with meta-analysis. Journal of Personality, 48(4), 449-472.
- DuBois, D. L., Holloway, B. E., Valentine, J. C., & Cooper, H. (2002). Effectiveness of mentoring programs for youth: A meta-analytic review. American Journal of Community Psychology, 30(2), 157-197.
- Peterson, J. L., & Shibley Hyde, J. (2010). A meta-analytic review of research on gender differences in sexuality, 1993–2007. Psychological Bulletin, 136(1), 21-38.
- Schwandt, T. A. (2000). Meta-analysis and everyday life: The good, the bad, and the ugly. American Journal of Evaluation, 21(2), 213-219.
- Wilson, S. J., & Lipsey, M. W. (2000). Wilderness challenge programs for delinquent youth: a meta-analysis of outcome evaluations. Evaluation and Program Planning, 23(1), 1-12.
Data sets and supplementary materials
- Effect Size Substantive Interpretation Guidelines
- Effect Size Calculator for Standardized Mean Difference Type Effects (Wilson, 2001)
- Meta-Analysis with Means, Binary Data, and Correlations
- Campbell Collaboration Effect Size Calculator (Wilson, n.d.)
- American Psychological Association Meta-Analysis Reporting Standards
- Sample Code Book
- Data Sets 1-6
- Data Sets for Heterogeneity Analysis
- Data Set for Subgroup Analysis
Homework and projects
Lecture notes
- Lecture 1PPTX
- Lecture2 PPTX
- Lecture 3 PPTX PDF
- Lecture 4 PPTX PDF
- Lecture 5 PPTX PDF
- Lecture 6 PPTX PDF
- Lecture 7 PPTX PDF
- Lecture 8 PPTX PDF
- Lecture 9 PPTX PDF
- Lecture 10 PPTX PDF