Evaluation of the Kalamazoo Promise

This is an outline of work that the College of Education is conducting regarding the Kalamazoo Promise, a scholarship program funded by anonymous donors that was announced in November 2005. Since its announcement, there has been substantial interest in the potential effects of the universal scholarship. Because of this interest and because other communities are replicating or planning to replicate the program, an evaluation that examines the impact of the scholarship program is critical. The purpose of the evaluation is to determine whether the Kalamazoo Promise--through its transformative effect on the education culture within the school district--improves the progress of students through their K-12 experience and better prepares them for entering a postsecondary education program.

Some imagine that at least 4 to 5 years will be required to measure the anticipated long-term outcomes (i.e., increased access and success in post-secondary education). Our evaluation, however, maps out and tracks the impact of the program which can be measured with specific short-term and intermediate outcomes that are linked to the long-term outcomes. Aside from anticipated outcomes, our evaluation will also consider unanticipated outcomes stemming from the inception of the scholarship.

Data will be collected using five distinct activities:

  1. surveys of middle school and high school students,
  2. surveys of teachers/staff,
  3. interviews of key informants,
  4. review of public and district documentation, and
  5. regression analysis of school-level data relative to demographically and geographically similar schools in the state.

In addition to this, the W.E. Upjohn Institute for Employment Research plans to conduct student level analyses to examine the impact of the Promise on student-level indicators.

The evaluation is both formative and summative in nature. Tracking and monitoring short-term and intermediate outcomes requires that we partner with the school district and other key stakeholders groups to collect data. Based on the data we collect throughout the duration of the evaluation, we are preparing user-friendly formative feedback that can be used by the district to monitor the progress, identify obstacles, and inform changes that need to occur. This is, essentially, the formative aspect of the evaluation.

The summative aspect refers to the strong base of evidence we expect to have at the end of the evaluation that will help us to determine if a universal scholarship program, involving the short-term and intermediate outcomes as well as some early indications of progress toward the long-term outcome, is able to cause systemic change and improvement in a district that has been known to struggle with both education attainment and education achievement. The funding and support this evaluation received from the U.S. Department of Education is due to the interest of policy makers and a litany of communities across the nation that are replicating or planning to replicate the Promise scholarship program. The summative evidence from our evaluation will help to determine whether this is a reform model worthy of replication.

 

 

kalamazoo promise, kalamazoo, evaluation, research, school reform, systemic reform, systemic change, urban school
 

Kalamazoo Promise Evaluation
Western Michigan University
Kalamazoo MI 49008-5283 USA
(269) 387-3883 | (269) 387-3696 Fax
gary.miron@wmich.edu