Description

Cultural competence has been defined as “a set of congruent behaviors, attitudes and policies that come together in a system, agency or among professionals, and enables effective work in cross-cultural situations” (CDC, 2014; Sen Gupta et al., 2004). Culturally competent evaluation places culture at the center (Hopson, 2003) of evaluation theory, measures, analysis and practice (Endo Inouye et al. 2005). It also calls attention to the social location of the evaluator (Hopson, 2003), and “requires an awareness of self, reflection on one’s own cultural position, awareness of other’s positions, and the ability to interact genuinely and respectfully with others” (AEA, 2011).

Unique Contributions

This approach is unique in that it centers culture in evaluation. It emphasizes the need to acknowledge that evaluators reflect their own biases in their practice of evaluation. Many other evaluation approaches start with or assume cultural competence. For example, it is more difficult, if not impossible, to be culturally responsive if the team is not culturally competent or culturally fluent (Sen Gupta et al., 2004).

History

Hopson (2003) describes two strands of culturally competent evaluation: international and national. The roots of these strands were planted in the mid-1980s and early 1990s, as evidenced by two volumes of New Directions for Program Evaluation edited by Michael Patton and Anne-Marie Madison, respectively. In 1985, in reference to evaluation in international contexts, Patton posed the question: “What happens when we export the ideas, concepts, models, methods and values of evaluation to other countries and cultures?” Thus began a conversation about the role of culture in evaluation. In 1992, Madison called for the inclusion of multicultural peoples and perspectives in the design and development of evaluations in the United States. The idea of culturally competent evaluation continued to evolve through the 1990s and 2000s, with heavy influences and contributions from the fields of psychology and education and gained support in the 2000s from the National Science Foundation. However, the American Evaluation Association (AEA) didn’t adopt or publish a public statement on cultural competence in evaluation until 2011 (AEA, 2011). AEA’s public statement continues to be the standard for evaluators to conduct culturally competent evaluations.

PRINCIPLES

Promotion of equity and social justice; attendance to issues of power

A culturally competent evaluation should pay attention to “how race, power, inclusion, politics, and privilege may be affecting the context, and discuss their relevance to the program” (CDC, 2014). Culturally competent evaluator(s) “represent the values and standards of marginalized stakeholders [and] use their power to promote equality and self-determination” (AEA, 2011). However, the culturally competent evaluation approach has been critiqued when power is not addressed. Thomas et al. (2018) argue that “the emphasis on the multiple dimensions of culture has been inadequate in addressing many of the systemic racial issues related to social programs… attention to cultural responsiveness in evaluation has stalled thoughtful deliberation of how we could integrate issues of race and racism in our work.”

Engagement of partners and community members, particularly those with less social power, during all phases of the evaluation

Culturally competent evaluation requires engaging stakeholders that reflect the diversity of the community (in terms of demographics, beliefs, institutions, ideologies, etc.) and working to ensure these stakeholders can participate fully in the evaluation (CDC, 2014). When identifying key stakeholders, it may be helpful to talk to community leaders or others who can serve as bridge builders or cultural translators. These individuals can help the evaluator become familiar with cultural norms (Lee, 2007) and help “assess the community’s readiness and willingness to address the issues that the evaluation may raise” (CDC, 2014). A smaller group of key stakeholders may serve on an advisory team that is able to bring in a larger group of stakeholders as needed. It is important to build trust with all stakeholders. This can be achieved by talking with stakeholders “about the evaluation, why it is being done, and how the community can expect to benefit” (CDC, 2014). Defining clear roles and setting ground rules can also aid in establishing equality among stakeholders and evaluators. Once a group of stakeholders has been identified and prepared to participate, they can ensure that evaluation questions reflect the values of their community and that their input shapes decisions, judgments, interpretations and recommendations (CDC, 2014).

Composition of evaluation team and reflection on assumptions and biases

Assessment of cultural self-awareness is important for effective communication and establishment of trust and respect with stakeholders (CDC, 2014). Culturally competent evaluators reflect on their backgrounds and how those may shape their assumptions and frames of reference for evaluation (Hopson, 2003). They are “willing to engage in a dialogue about how culture, social identity, and privilege and power affect them personally and in their work” (Lee, 2007), and “work to avoid reinforcing cultural stereotypes and prejudice” (AEA, 2011).

Consideration of cultural and historical contexts and different worldviews

When conducting culturally competent evaluations it is important to “understand the program’s social, cultural, and historical contexts as well as the culture of the program itself” (CDC, 2014). Evaluators should familiarize themselves with culturally determined mores, values, and ways of knowing in the population being evaluated (AEA, 2011). This may require “deliberately setting aside time and resources in the evaluation timeline and budget to learn about differences and similarities” among stakeholders, as well as between the evaluator and stakeholders, and designing processes that take these similarities and differences into account (Lee, 2007).

Intentional methods and thoughtful data collection

The CDC (2014) notes that culturally competent evaluations should be sensitive to preferences for nonlinearity in logic model design, integrate multiple perspectives when determining what counts as credible evidence, rely on culturally appropriate data collection instruments, consider cultural and linguistic needs when planning data collection, and ensure that data collection processes are respectful of cultural mores (e.g., communication styles, conceptualizations of time, social hierarchies).

Intentional analysis and inclusive interpretation

Culturally competent evaluators “carefully examine the data to understand how contextual conditions and structural inequities affect the outcomes” (Lee, 2007). Mixed methods allow for the detection of patterns of statistical significance through quantitative analysis and contextual understanding of the patterns through qualitative analysis (Lee, 2007). “Culture serves an intrinsic role in influencing how we define categories for organization purposes, classify and interpret data, and determine which comparisons have meaning. Therefore, engaging stakeholders during the analysis is essential” (CDC, 2014). Culturally competent evaluators also consider the intersection of multiple cultural and social identities (Thomas & Campbell, 2020).

Accessible and actionable evaluation findings

The dissemination of evaluation results should be tailored to stakeholder needs and assessment of risks and benefits (Lee, 2007). Deliverables of culturally competent evaluations target multiple audiences and are creative in presentation style, product, language, tone, and graphics (CDC, 2014; Endo Inouye et al., 2005).