Overview
Principles
Click on each principle for more info
Composition of evaluation team and reflection on assumptions and biases
Assessment of cultural self-awareness is important for effective communication and establishment of trust and respect with stakeholders (CDC, 2014). Culturally competent evaluators reflect on their backgrounds and how those may shape their assumptions and frames of reference for evaluation (Hopson, 2003). They are “willing to engage in a dialogue about how culture, social identity, and privilege and power affect them personally and in their work” (Lee, 2007), and “work to avoid reinforcing cultural stereotypes and prejudice” (AEA, 2011).
Social investments often target populations of color and yet are evaluated by white evaluators (Public Policy Associates, 2015). Culturally responsive evaluation (CRE) argues that data collection, quality, and interpretation can be improved when evaluators share a lived experience with program participants (Frierson et al., 2002; Hopson, 2009; Hood et al., 2015). However, this does not mean that racial congruence automatically leads to cultural congruence (Frierson et al., 2002). A diverse evaluation team allows for multiple perspectives in the evaluation (Public Policy Associates, 2015) and genuine connection with the local context (Hood et al., 2015). All evaluators should critically reflect on their own personal cultural preferences and biases (Hood et al., 2015; Hopson 2009; Public Policy Associates, 2015) and “make a conscious effort to restrict any undue influence they might have on the work” (Frierson et al., 2002).
Culturally responsive and equitable evaluation (CREE) advocates for diverse evaluation teams that provide multiple voices and perspectives. “Evaluators must recognize that different life experiences lead to different views of the world, and that linguistic, historical, and socioeconomic differences can be contributors and/or barriers to understanding” (Public Policy Associates, 2015). Suggestions include hiring staff from diverse backgrounds and life experiences. To understand the community perspective, evaluation team members should include evaluators who share similar experiences. “For areas where the evaluation team does not have shared life experiences with the priority population and cannot compensate for that fact, be very conscious of these differences and how they might impact key evaluation findings” (Public Policy Associates, 2015). CREE also stresses the importance of evaluators’ awareness of their own biases and recommends cultural competence training (Anderson et al., 2020).
Empowerment evaluation is designed to be community and stakeholder owned, in which communities and partners self-determine processes and outcomes, and measure against those. Thus, the composition of the evaluation team is comprised primarily of community partners/stakeholders and an evaluation coach/facilitator. Empowerment evaluation is described as being driven by group process — the group serves as its own review and accountability group. The group learns from each other, serving as their own peer review group (Fetterman & Wandersman, 2004). Empowerment evaluation is designed to empower stakeholders to be able to do their own evaluation, capacity building, and thinking through of the values (and considerations) that underlie the evaluation.
Equitable evaluation in both the international and U.S. domestic contexts focuses on having a diverse and inclusive team to reflect the situations and contexts. “Selecting a strong team to conduct an equity-focused evaluation is a key step in a successful evaluation process. A good team must have an appropriate mix of skills and perspectives” (Bamberger & Segone, 2011). When assembling teams, it is important to “identify and include individuals with shared experiences related to the issue at hand when considering evaluation design and implementation, meeting structures, time durations, and locations” (Dean-Coffey et al., 2014). The practice of the Equitable Evaluation Framework™ (EEF) “calls upon individuals [evaluators] to understand how race and ethnicity operate with respect to equity and how race and ethnicity intersect with other socially defined characteristics, such as sexuality, class, nationality, and age“ (Dean-Coffey et al., 2014) as well as reflection on “our history because it sheds light on the implicit and explicit beliefs, values, and intentions that frame our current approaches and frameworks“ (Dean-Coffey, 2018). Three mindset shifts have been identified as important for practitioners of the Equitable Evaluation Framework™: moving from doing to being, from scarcity to abundance, and from fixed to growth (EEI & GEO, 2021). These mindset shifts are key to being able to be in service of equity.
Evaluators are not viewed as unbiased or detached from the evaluation process. Transformative evaluators expect that “their values in regard to social justice and human rights will influence the process and outcomes of their work” (Mertens, 2012). Additionally, to do their work well, evaluators must be aware of culture and power dynamics and be respectful. “To know realities, it is necessary to have an interactive link between the researcher and the participants in a study. Knowledge is socially and historically located within a complex cultural context. Respect for culture and awareness of power relations is critical (epistemological assumption). Respect is critically examined in terms of the cultural norms of interaction within a community and across communities (axiological assumption) (Mertens, 2007).
© 2022 SLP4i and The Colorado Trust, authored by Katrina Bledsoe, Felisa Gonzales, and Blanca Guillen-Woods. This work is protected by copyright laws. No permission is required for its non-commercial use, provided that the authors are credited and cited.
For full citation use: Bledsoe, K., Gonzales, F., & Guillen-Woods, B*. (2022). The Eval Matrix©. Strategic Learning Partners for Innovation https://slp4i.com/the-eval-matrix.
*These authors contributed equally to this work with support from the Annie E. Casey Foundation and The Colorado Trust.
The Eval Matrix site designed by KarBel Multimedia