Description

Transformative evaluation was developed by Donna Mertens and was directly informed by her personal commitment to social justice and her personal experiences working with the deaf and hard-of-hearing communities. “The transformative paradigm is applicable to the study of the power structures that perpetuate social inequities” (Mertens, 2012). It “focuses primarily on viewpoints of marginalized groups and interrogating systemic power structures through mixed methods to further social justice and human rights” (Mertens & Wilson, 2019). Mertens, who is the primary scholar of this approach, has published extensively on the transformative paradigm, which is grounded in four key philosophical assumptions:

– The axiological assumption holds that evaluation should be performed in the service of social justice (Mertens, 2016), which is most likely to occur when the importance of being culturally respectful is acknowledged; there is an explicit promotion of social justice; human rights are furthered; inequities are addressed; reciprocity is achieved; and community strengths and resilience are recognized (Mertens & Wilson, 2019).

– The ontological assumption holds that different versions of reality dependent on social positionality exist (Mertens, 2016); it “interrogates versions of reality on the basis of power inequities and the consequences of accepting one version of reality over another” (Mertens & Wilson, 2019).

– The epistemological assumption holds that differences in power impact the ability to accurately identify problems and solutions (Mertens, 2016); “knowledge is neither absolute nor relative. Rather, it is constructed within a context of power and privilege with consequences attached to which version of knowledge is given privilege” (Mertens & Wilson, 2019).

– The methodological assumption holds that mixed methods can be used to be responsive to diverse stakeholder groups and capture the complexity of the phenomenon under study in ways that contribute to social transformation (Mertens 2016); “methodological decisions are aimed at determining the approach that will best facilitate use of the process and findings to enhance social justice; identify systemic forces that support the status quo and those that will allow change to happen; and acknowledge the need for a critical and reflexive relationship between the evaluator and the stakeholders” (Mertens & Wilson, 2019).

Unique Contributions

Transformative evaluation begins with ethics and values to commit to justice and equity. It requires evaluators to bring a critical lens to whose reality is being voiced, whose reality is being privileged, and then bring to light the related consequences. In order to do that, evaluators must be thoughtful of their methodological approaches to be able to include data that reflects the realities of those with less power, including intersectional experiences. Transformative evaluation is action-oriented and designed to support transformative change for the communities that are participating in the evaluation.

History

Mertens was influenced by a strong sense of moral values, early discussions of ethics in evaluation, and the civil rights movement. She developed the transformative approach as a way to transform society through evaluation. It was developed to conduct evaluation guided by clear ethics, values, and a commitment to social equity. Further, Mertens’ experience working with multiple communities and contexts around the world, including working with members of deaf and hard-of-hearing communities, highlighted the need to identify salient characteristics within a specific context and community that served as the basis for disparity within that community. Transformative evaluation was designed to serve as an umbrella approach that could be applied in multiple contexts to identify the root of disparities in the specific context, engage and amplify the voice of those most negatively impacted, and then bring forward solutions identified by that community to address the inequities. The transformative paradigm “includes evaluators who develop theoretical frameworks based on cultural responsiveness; race/ethnicity; on human rights, feminist, disability rights, deafness, postcolonial/Indigenous, and queer theories; and on other marginalized communities” (Mertens & Wilson, 2019).

PRINCIPLES

Promotion of equity and social justice; attendance to issues of power

Power is a central issue that must be addressed at each stage of a transformative evaluation (Mertens, 2007). Evaluators who subscribe to the transformative paradigm “consciously and explicitly position themselves side by side with the less powerful in a joint effort to bring about social transformation” (Mertens, 2020). The transformative evaluator focuses on “unequal distributions of power and the resultant oppression of subjugated groups,” and “a preset goal of the research is to empower participants to transform the status quo and emancipate themselves from ongoing oppression” (Mertens, 2012). “Evaluators have a responsibility to make visible the dynamics of discrimination and oppression that are relevant in the evaluation context” (axiological assumption) (Mertens, 2016). “The transformative paradigm has relevance for people who experience discrimination and oppression on [the basis of any] characteristics that are associated with less access to social justice” (Mertens, 2010).

Engagement of partners and community members, particularly those with less social power, during all phases of the evaluation

“The evaluator has an ethical responsibility to engage with stakeholders to examine the assumptions about the problem, solution, and evaluation methods in order to increase the potential for social change” (Mertens, 2016). “Community participation is needed at the beginning, throughout, and at the end of each research study” (Mertens, 2007). “The evaluation focus, purpose, design, implementation, and utilization should be developed through a cooperative process between the evaluator and community members” (Mertens, 2012). Benefits of an inclusive approach include “aiding in impartiality, gaining new information, and improving the sensitivity of evaluation” (Mertens, 1999). “Evaluators need to establish trusting relationships with the full range of stakeholder groups in order to obtain an accurate picture of the problem under study” (epistemological assumption) (Mertens, 2016). Coalitions are one means of ensuring community members inform the evaluation process and are positioned to support sustainability when the evaluation ends.

Composition of evaluation team and reflection on assumptions and biases

Evaluators are not viewed as unbiased or detached from the evaluation process. Transformative evaluators expect that “their values in regard to social justice and human rights will influence the process and outcomes of their work” (Mertens, 2012). Additionally, to do their work well, evaluators must be aware of culture and power dynamics and be respectful. “To know realities, it is necessary to have an interactive link between the researcher and the participants in a study. Knowledge is socially and historically located within a complex cultural context. Respect for culture and awareness of power relations is critical (epistemological assumption). Respect is critically examined in terms of the cultural norms of interaction within a community and across communities (axiological assumption) (Mertens, 2007).

Consideration of cultural and historical contexts and different worldviews

“The use of a social justice lens highlights the need for careful contextual analysis as the starting point for evaluations that are focused on social transformation designed to improve human rights” (Mertens, 2016). Contextual analyses are an essential stage of the evaluation. Evaluators are charged with acknowledging “the historical and social location of knowledge, combined with an understanding of the effect of a legacy of discrimination and oppression. The evaluator’s role is to bring to visibility historical knowledge of this nature, link it to the social positioning of those who create the knowledge, and critically examine the consequences of accepting one version of reality over another” (Mertens, 2016). The transformative ontological assumption holds that “there are multiple realities that are socially constructed” (Mertens, 2007); that is, different realities can emerge and be experienced based on one’s lived experience. Evaluation should explore different versions of reality for people from different social positionalities in “determining the reality that holds potential for social transformation and increased social justice” (Mertens, 2007).

Intentional methods and thoughtful data collection

Evaluators should use mixed methods to capture the complexity of the problem under study (Mertens, 2020). “The transformative methodological assumption includes the concepts of inclusion of the full range of stakeholders in the process of decision-making about the evaluation methods, conduct of contextual analyses that identify cultural factors and issues of power as a basis for building trusting relationships, use of mixed methods to capture the cultural complexity and the data necessary to be appropriately responsive and inclusive of diverse stakeholders, building on community assets, and designing the evaluation to challenge assumptions that impede progress toward positive social change. … a mixed methods approach has the potential to provide a fuller picture of the complexities of problems, as well as the design, implementation, and determination of the effectiveness of solutions” (Mertens, 2016). Mertens proposes a cyclical model of mixed-methods research “as a way of involving the community in research decisions and the collection of data that can be used for social justice purposes” (Mertens, 2007; Mertens, 2009).

Intentional analysis and inclusive interpretation

Most germane to analysis is the transformative ontological assumption which deals with the nature of reality. This assumption “holds that there are diversities of viewpoints with regard to many social realities, but we need to place those viewpoints within a political, cultural, and economic value system to understand the basis for the differences. And then, we need to struggle with revealing those multiple constructions, as well as with the decisions about privileging one perspective over another” (Mertens, 1999). “We are led to ask questions such as, ‘Whose reality is privileged in this context?’ ‘What is the mechanism for challenging perceived realities that sustain an oppressive system?’ ‘What are the consequences in terms of who is hurt if we accept multiple versions of reality or if we accept the ‘wrong/privileged’ version?’” (Mertens, 2010).

Accessible and actionable evaluation findings

The transformative paradigm means an “explicit connection is made between the process and outcomes of research and furtherance of a social justice agenda” (axiology assumption) (Mertens, 2007). Ensuring that evaluation can and will further a social justice agenda requires consideration of the products that will be developed and how they will be utilized. “The challenge is to approach the evaluation in a way that builds in a greater likelihood of utilization. However, the challenge goes further, and asks, ‘What is the evaluator’s role in facilitating utilization in a way that holds greater promise that the social ills that formed the basis for the need for the program can actually be ameliorated?’ ‘How can we insure or increase the probability that the bridge between evaluation findings and social transformation can be crossed?’ Transformation can only occur if this information is used to inform policies that effectively address the inequities that create the need for social programs” (Mertens, 1999).