Exams and Dissertation

As a student in the interdisciplinary Ph.D. in evaluation program at Western Michigan University, you will complete these requirements.

Doctoral comprehensive examination procedure

The interdisciplinary Ph.D. in evaluation program comprehensive examination procedure requires successful completion of the following:

1. An approved program plan of study.
2. A portfolio documenting competency.
3. Presentation of a poster demonstrating competency.
4. An oral examination demonstrating competency.

Details regarding the program plan of study are available from the program director or coordinator upon request. The program plan of study needs to be approved and signed. The expected competency details can be found in the competency self-assessment instrument (available from the program director or coordinator upon request).

Program plan of study

A minimum of 90 graduate credit hours is required for completing the program, including credits within the following domains:

• Cognate (18-21 credit hours)
• Research methods (12-18 credit hours)
• Evaluation (35-39 credit hours)
• Professional field experience (nine credit hours)
• Doctoral dissertation (minimum of 12 credit hours; not started until successful passing of the comprehensive examination)

Portfolio

The portfolio should represent your mastery of field specialization, breadth of field specialization, research productivity and practical experience. The three domains (i.e., cognate, research methods, evaluation) should be supported with documentation, including, but not limited to the following:

• Curriculum vitae.
• Autobiographical statement of career and professional goals.
• Personal statement of professional evaluation qualifications and competencies.
• Program plan of study (approved and signed by all relevant parties).
• Artifacts from coursework.
• An initial self-assessment of required competencies (i.e., prior to beginning the program).
• A final self-assessment of required competencies (i.e., upon completion of the program of study); For each competency, provide evidence of meeting the required competencies and provide a plan, if a competency is not fully met, for attaining mastery of the required competency.
• Evidence of evaluation or research productivity (e.g., publications, presentations, technical reports).

Poster presentation

You will demonstrate meeting the competencies through a display of a sample of your work (this may include work completed with others; however,you should have played a major role in the work). The poster presentation should communicate your readiness to advance to the dissertation stage. The poster will be displayed in a public location (determined by the program director) where faculty, staff and students will be encouraged to ask questions and engage you in conversations. It is expected that you will accompany your poster display and answer questions for approximately two hours. Faculty, staff and students will be asked to complete a brief rubric regarding your poster presentation and competency. Poster presentations and oral examinations will be scheduled at a minimum in May/June of each year. If you wish to proceed to dissertation stage, you will be expected to present your poster and participate in the oral examination on the selected dates (which vary from year to year).

Oral examination

Upon successful completion of your program plan of study, portfolio and poster presentation, you will participate in a 1½ to 2 hour oral examination with the program director and two other committee members. The examination will generally cover your knowledge of evaluation, not methods or cognate (though these may arise as part of the examination process related to evaluation), with an emphasis on your theoretical and practical knowledge of evaluation. The oral examination also will be used to discuss your dissertation plan. The oral examination committee will consider all evidence (i.e.,your program plan of study, portfolio, poster presentation and oral examination) in adjudicating whether or not you meet the minimum standards for successful completion of the comprehensive examination for the program.

Comprehensive examination policy

Each element of the examination procedure described above (i.e., portfolio, poster presentation and oral examination) will be graded as exceeds standards, meets standards, approaches standards or fail by the examiners. A single examiner’s grade of fail constitutes a failure of the examination. If you receive a grade of fail on any element of the examination you will be dismissed from the program. If you receive an approaches standards on any element of the examination, you will be required to retake the entire examination at the next available date. If you receive a fail or approaches standards on the portfolio or poster presentation, you will not advance to the oral examination.

To dispute or appeal an examination grade, contact the WMU Ombudsman.

Doctoral dissertation

Prospectus

The dissertation prospectus is a short, 8-10 page description of the problem statement, method and contribution to evaluation theory, method or practice that precedes a formal dissertation proposal. Essentially, the dissertation prospectus should provide the committee chair and other committee members a general understanding of the scope and nature of the problem to be investigated.

Proposal

The dissertation proposal includes Chapters I and III if using the traditional five-chapter dissertation or, if using the three-paper dissertation, a clear presentation of the three proposed papers. Unlike the dissertation prospectus, the dissertation proposal is more well-developed and should be approved by the dissertation chair prior to defense. Dissertation proposals are presented and defended to the committee members. The dissertation committee then determines whether the proposal is adequate (or not) and whether the proposed dissertation research may proceed (or not) and whether modifications to the proposal are necessary.

Five-chapter dissertation

The following is a checklist of items which are typically included in a traditional five-chapter dissertation. Not all of the suggested categories are necessary or appropriate, and the items within chapters may vary somewhat. They are intended to serve as a guide. As a point of reference when using this checklist, the overarching standard for an interdisciplinary Ph.D. in evaluation dissertation is that it should be an original, significant contribution to evaluation theory, methodology or practice.

Chapter I: Problem Statement

Introduction

• Background of the problem (e.g., educational trends related to the problem, unresolved issues, social concerns).
• Statement of the problem situation (basic difficulty—area of concern, felt need).
• Purpose of the study (goal oriented)—emphasizing practical outcomes or products.
• Questions to be answered or objectives to be investigated.
• Conceptual or substantive assumptions (postulates).
• Rationale and theoretical framework (when appropriate).
• Delineation of the research problem (explication of relationships among variables or comparisons to be considered).
• Statement of hypotheses (conceptual rendition subsequently followed by operational statements in Chapter I or Chapter III).
• Importance of the study—may overlap with statement of the problem situation.
• Definition of terms (largely conceptual; operational definitions may follow in Chapter III).
• Scope and delimitations of the study (narrowing of focus).
• Outline of the remainder of the thesis, dissertation, or prospectus or proposal.

Chapter II: Review of the Related Literature

• Organization of the chapter—overview.
• Historical background (if necessary).

Purposes to be served by review of research literature:

• Acquaint reader with existing studies relative to what has been found, who has done work, when and where latest research studies were completed, and what approaches involving research methodology, instrumentation and statistical analyses were followed (literature review of methodology sometimes saved for chapter on methodology.
• Establish possible need for study and likelihood of obtaining meaningful, relevant and significant results.
• Furnish from delineation of various theoretical propositions a conceptual framework affording bases for generation of hypotheses and statement of their rationale (when appropriate).

In some highly theoretical studies, the Review of the Literature chapter may need to precede The Problem chapter so that the theoretical framework is established for a succinct statement of the research problem and hypotheses. In such a case, the advance organizer in the form of a brief general statement of the purpose of the entire investigation should come right at the beginning of the Review of Related Literature chapter.

Sources for literature review

• General integrative reviews that relate to the problem situation or research problem.
• Specific books, monographs, bulletins, reports and research articles—preference shown in most instances for literature of the last ten years.
• Unpublished materials (e.g., dissertations, theses, papers presented at professional meetings).
• Selection and arrangement of literature review often in terms of questions to be considered, hypotheses set forth, or objectives of specific purposes delineated in the problem chapter.

Summary of literature reviewed

Chapter III: Method

• Overview (optional).
• Description of research methodology or approach (e.g., experimental, quasi-experimental, nonexperimental).
• Research design (if relevant, describe independent, dependent and classificatory variables and sometimes formulate an operational statement of the research hypotheses in null form so as to set the stage for an appropriate research design permitting statistical inferences).
• Pilot studies or pretesting (as they apply to the research design, instruments, data collection techniques and characteristics of the sample).
• Selection of subjects (this is concerned with sample and population).
• Instrumentation (tests, measures, observations, scales and questionnaires).
• Procedures (e.g., instructions to subjects or distribution of materials).
• Data collection and recording.
• Data processing and analysis.
• Methodological assumptions.
• Limitations.
• Possible restatement of conceptual hypotheses from problem chapter in operational form relative to instrumentation and experimental procedure or design followed (operationally stated hypotheses can also be put in null form to furnish an optional third set of hypotheses amenable to statistical testing)—if not done elsewhere.
• Summary (optional).

Chapter IV: Results (or findings)

• Findings are presented in tables or charts when appropriate.
• Findings reported with respect to furnishing evidence for each question asked or each hypothesis posed in problem statement.
• Appropriate headings are established to correspond to each main question or hypothesis considered.
• Factual information kept separate from interpretation, inference, and evaluation (one section for findings and one section for interpretation or discussion).

In certain historical, case study and anthropological investigations, factual and interpretive material may need to be interwoven to sustain interest level, although the text should clearly reveal what is fact and what is interpretation.

• Separate section often titled Discussion, Interpretation or Evaluation ties together findings in relation to theory, review of literature or problem statement.

Chapter V: Summary, Conclusions and Recommendations

• Brief summary of everything covered in first three chapters and in findings portion of Chapter IV.
• Conclusions (so what of findings; often the hypotheses restated as inferences with some degree of definitive commitment and generalizability).

Three-paper dissertation

Guidelines:

• The three-article dissertation incorporates three or more stand-alone articles and an abstract that synthesizes the articles, as well as an introduction (Chapter I) and a conclusion (Chapter V, assuming three articles are presented).
• The articles must describe original research.
• The committee must deem the articles of publishable quality.
• At least one of the three papers must be submitted for publication prior to your defense.
• Only one paper of the three papers can be published at the time of your defense.
• You must be first author on all articles. As a first author, you are responsible for development and articulation of a concept or idea for research, development of a proposal to pursue this idea, development of a research design, conducting research and analysis, writing the manuscript, designing an intervention or assessment (if relevant), and interpreting results. The role of the coauthors must be presented and approved by all members of the dissertation committee.
• Research and consider possible copyright issues that may arise when trying to publish your dissertation. Possible steps that may need to be taken to ensure that the articles can be published include opting to NOT make your dissertation available to the public or securing copyright permission before submitting your dissertation to WMU’s Graduate College
• A certain amount of overlap is acceptable among the three articles. For example, portions of the literature review may need to be cited in the various articles because they are central to the historical background of the study. Redundancy can be reduced by citing your own work. Self-plagiarism (i.e., the failure to cite one’s own previously written work or data in a new written product) is prohibited.