Evaluation Café
Since the 1980s, the Evaluation Café has been a venue for spirited dialogue about the cutting edge of evaluation.
The Evaluation Café features discussions, debates, and presentations about evaluation over the lunch hour. Café presenters introduce new ideas, solicit feedback on real-world evaluation issues, and share recent evaluation work.
Evaluation Café is pleased to offer both in-person and virtual options. Join us for lunch and a thought-provoking conversation led by innovative minds in our field in 4410 Ellsworth or via Zoom.
Café Schedule
We hope you'll join us for our exciting fall lineup!
All presentations begin at 12:05 p.m. EST.
Modern Reporting Strategy - Thinking Beyond the PDF
September 6, 2023
Do any of these describe you?
- Are you a program evaluator, researcher, data analyst, or someone else who regularly shares data and evidence with others?
- Do you care enough about your work to actually share it with others?
- Do you want to create better reports that are more accessible and reach more people?
- Do you feel like your reports are either too long or too short, with absolutely no in between?
- Does the idea of creating more reports just sound like tons more work?
- Not sure whether you should create infographics, dashboards, slidedocs, or something else entirely?
Join Chris Lysy of freshspectrum.com, and author of The Reporting Revolution, as he discusses why we need to think beyond the PDF and walks you through building your own modern reporting strategy. As a bonus: everyone who attends will also get access to a free modern reporting strategy template you can use to start crafting your own process.
Supporting Criteria Specification in Practice: A Model for Surfacing Values and Shaping Lines of Inquiry
September 20, 2023
Evaluative criteria represent values about what a high-quality or successful intervention “looks like”. These implicit or explicit criteria direct evaluators’ lines of inquiry, including which evaluation questions are asked, data are collected and analyzed, and conclusions are reached and reported. Community members, program participants, staff, leaders, funders, and evaluators often hold varying values. Thus, evaluators are charged with the complex tasks of identifying relevant values, specifying appropriate criteria, and applying those criteria to direct inquiry. This presentation will introduce an empirically supported model of evaluative criteria developed to guide evaluation practice. Discussion will highlight how the framework can be used to support criteria specification, make criteria more explicit, broaden the range of values that shape evaluative inquiry, and clarify evaluation design and reporting. The presentation will also explore current research on evaluation to refine the model and deepen understanding of practice.
CRREE Makeover: Partnering with Commissioners of Evaluation to Refine Evaluation Methodology
Culturally Responsive and Racially Equitable Evaluation (CRREE) takes time to consider how diversity, assessment, inclusion, community engagement, and equity all shape what is possible for evaluation to be meaningful, accessible, and actionable for learning organizations and the communities we love (Mirror Group). In this presentation, Mindelyn Anderson PhD, Founder + Principal of Mirror Group, will walk through a real-life "CRREE Makeover" of social services program evaluation where Mirror Group partnered with nonprofit leadership, program administrators, the funder, and community members to design and conduct an evaluation to meet all partner needs.
Approaches to Design and Research for Language Justice
In this conversation on strategies for multilingual research, Laura Gonzales and Allison Corbett will challenge assumptions about how to conduct research with and for multilingual communities. Laura Gonzales will address the question of how to adequately assess linguistic needs within a community by going beyond traditional sources of demographic data and will discuss strategies for measuring the impact of multilingual initiatives through community engagement. Allison Corbett will introduce the framework of language justice as a means of holistically designing multilingual survey research and will share the approach to the 2021 re-design of the New York City Housing and Vacancy Survey as a case study.
Criteria are the heart of evaluation. Or are they?
As an evaluator, I’ve grown up on the “values branch" of the evaluation theory tree. To me, evaluating is all about assessing the value of something. What value means depends on the criteria we use. Criteria support the validity, credibility, and utility of an evaluation. Yet, research on evaluation suggests that most evaluators do not set criteria. Clients set criteria and evaluators use them (often implicitly). This is problematic and, in this talk, I will unpack why. Criteria are the heart of evaluation: a chance to ensure that evaluative processes center on what really matters. Criteria should be set purposefully with participatory input from those involved in and affected by an intervention. There are lots of ways to do this and I will share a few examples. After making a case for the importance of evaluative criteria, I will address two critiques: one from the evidence-based policy and practice movement and another emerging from systems change and transformation. These critiques, in different ways, raise the question of whether criteria really are the heart of evaluation - a question I believe is worth revisiting as a field.
Decolonizing the JMDE Publication Process: Courageous Conversations and Equity Engaged Activities
In July 2023, The Evaluation Center at WMU partnered with the International Evaluation Academy to publish a special issue of the Journal of MultiDisciplinary Evaluation titled “Decolonizing Evaluation: Towards a Fifth Paradigm”. This was considered a watershed for the International Evaluation Academy - the first science output, and on such an important topic. The special issue was edited by Bagele Chilisa and Nicky Bowman, with managerial support from Michael Harnar, current co-executive editor of JMDE. The collaboration among the partners was deep and at times intimate, as they negotiated how best to respectfully dive into the challenging topic of “decolonizing evaluation”. As Ian Goldman wrote in his Foreword, this was to be “an important contribution to rethinking how evaluation can be fit for purpose in a world of polycrises, where the traditional Western growth models have led us to the brink of catastrophe. We need different ways of looking at the world, and learning from indigenous and decolonized approaches will be key in us garnering the wisdom to learn and transform how we use evaluation in the service of humanity and Nature, rather than for continued exploitation and despoilation.” (p. 1). In this Eval Café presentation, Bagele, Nicky, and Michael will reflect on the collaboration that brought this important contribution to the open-access evaluation canon and discuss some of the numerous values-laden, decolonized, and cultural conversations they had in the process.