Overview
Principles
Click on each principle for more info
Engagement of partners and community members, particularly those with less social power, during all phases of the evaluation
Culturally competent evaluation requires engaging stakeholders that reflect the diversity of the community (in terms of demographics, beliefs, institutions, ideologies, etc.) and working to ensure these stakeholders can participate fully in the evaluation (CDC, 2014). When identifying key stakeholders, it may be helpful to talk to community leaders or others who can serve as bridge builders or cultural translators. These individuals can help the evaluator become familiar with cultural norms (Lee, 2007) and help “assess the community’s readiness and willingness to address the issues that the evaluation may raise” (CDC, 2014). A smaller group of key stakeholders may serve on an advisory team that is able to bring in a larger group of stakeholders as needed. It is important to build trust with all stakeholders. This can be achieved by talking with stakeholders “about the evaluation, why it is being done, and how the community can expect to benefit” (CDC, 2014). Defining clear roles and setting ground rules can also aid in establishing equality among stakeholders and evaluators. Once a group of stakeholders has been identified and prepared to participate, they can ensure that evaluation questions reflect the values of their community and that their input shapes decisions, judgments, interpretations and recommendations (CDC, 2014).
“Stakeholders play a critical role in all evaluations, especially culturally responsive ones, providing sound advice from the beginning (framing questions) to the end (disseminating the evaluation results)” (Frierson et al., 2002). It is important to acknowledge that “people are at the center” of culturally responsive evaluation(CRE) and they know what they value and need (McBride, 2018). This approach recommends that evaluators “develop a stakeholder group representative of the populations the project serves, assuring that individuals from all sectors [and varying levels of power, status, and resources (Hood et al., 2015)] have the chance for input” (Frierson et al., 2002). One important way of doing this is to listen and hear the questions of stakeholders so that they are incorporated into the evaluation design (Frierson et al., 2002). As in culturally competent evaluation, it is critical to establish a climate of trust and respect among all parties involved (Hood et al., 2015).
In keeping with the definition of culturally responsive and equitable evaluation (CREE), Lemos and Garcia (2020) assume that community members are partners and are an essential part of the evaluative process, from conceptualization to dissemination. They argue that the community must be involved in the change process, with data made available to community members in order to bring about change. Evaluators need to spend adequate time during the design phase to align their understanding, expectations, and strategies. Lemos and Garcia (2020) state that evaluators who work collaboratively with diverse Latinx and other diverse communities should ensure that these perspectives are integrated throughout the process and address unanticipated challenges.
In the best-case scenario in culturally responsive Indigenous evaluation (CRIE), the team would be led by Indigenous evaluators and stakeholders. In their article focusing on tribal populations’ relationship with federal governments, Waapalaneexkweew (Nicole Bowman, Mohican/Lunaape) and Dodge-Francis (2018) provided an example in which the evaluation team asked tribal stakeholders for help in guiding the activities of the evaluation. The project team used multiple strategies to receive varied and diverse stakeholder feedback throughout the study design, testing, and full data-collection process. This included meeting documents and formal administrative responses/reactions to the project and team, including production and dissemination of the final report.
Specifically, the 10 principles of empowerment evaluation are geared toward empowering the key stakeholder communities to conceptualize, lead, and perform their own evaluation work. Key principles include (1) community ownership – empowerment evaluation values and facilitates community control, and use and sustainability are dependent on a sense of ownership; (2) inclusion – empowerment evaluation invites involvement, participation, and diversity and contributions come from all levels and walks of life; (3) community knowledge – empowerment evaluation respects and values community knowledge; (4) and capacity building – empowerment evaluation is designed to enhance stakeholders’ ability to conduct an evaluation and to improve program planning and implementation (Fetterman et al., 2015).
Engagement of partners and community members, particularly those with less social power, is a key component of equitable evaluation. “Involvement in the design, planning, and implementation of the evaluation of those stakeholders (especially worst-off groups) directly affected by an intervention (be they the implementers or intended beneficiaries), is a fundamental principle of any equity-focused evaluation … guaranteeing stakeholder participation strengthens accountability, builds trust and agreement in the evaluation process, generates credibility, and can itself contribute to equity building” (Bamberger & Segone, 2011). In equitable evaluation, stakeholder participation can be facilitated by paying attention to group dynamics, development, needs, and effectiveness, as well as ensuring that materials are translated “into languages and formats designed to ensure that everyone can participate fully in the evaluation and plan to share data in meaningful ways to all populations” (Dean-Coffey et al., 2014).
“The evaluator has an ethical responsibility to engage with stakeholders to examine the assumptions about the problem, solution, and evaluation methods in order to increase the potential for social change” (Mertens, 2016). “Community participation is needed at the beginning, throughout, and at the end of each research study” (Mertens, 2007). “The evaluation focus, purpose, design, implementation, and utilization should be developed through a cooperative process between the evaluator and community members” (Mertens, 2012). Benefits of an inclusive approach include “aiding in impartiality, gaining new information, and improving the sensitivity of evaluation” (Mertens, 1999). “Evaluators need to establish trusting relationships with the full range of stakeholder groups in order to obtain an accurate picture of the problem under study” (epistemological assumption) (Mertens, 2016). Coalitions are one means of ensuring community members inform the evaluation process and are positioned to support sustainability when the evaluation ends.
© 2022 SLP4i and The Colorado Trust, authored by Katrina Bledsoe, Felisa Gonzales, and Blanca Guillen-Woods. This work is protected by copyright laws. No permission is required for its non-commercial use, provided that the authors are credited and cited.
For full citation use: Bledsoe, K., Gonzales, F., & Guillen-Woods, B*. (2022). The Eval Matrix™. Strategic Learning Partners for Innovation https://slp4i.com/the-eval-matrix.
*These authors contributed equally to this work with support from the Annie E. Casey Foundation and The Colorado Trust.
The Eval Matrix site designed by KarBel Multimedia