Department Environmental Social Sciences

Theory of Change (ToC) has been promoted as a useful tool in sustainability research for visioning, planning, communication, monitoring, evaluation and learning. It involves a mapping of steps towards a desired long-term goal supplemented with continuous reflection on how and why change is expected to happen in a particular context. However, there is limited reported experience with the development and application of ToCs in inter- and transdisciplinary research contexts. While some previous publications have focused on ex-post application, there has been little discussion about the process of developing and using ToCs in strategic planning and monitoring in large inter- and transdisciplinary research programs. This article reports challenges and lessons learned from the experience of developing and using ToCs in the inter- and transdisciplinary research program Wings (Water and sanitation innovations for non-grid solutions). Challenges include (1) managing time constraints, (2) balancing between concrete and abstract discussions, (3) ensuring diversity in group composition, (4) fluctuating between reservations and appreciation, and (5) fulfilling both service and science roles while leading the ToC process. The experience highlights the importance of alternating formal and informal interaction formats throughout the process, ensuring heterogenous group formation, involving early career scientists, being responsive to emergent needs and making the added value of developing and using ToCs explicit and tangible for all participants. Although these lessons are mainly derived from developing ToCs within the interdisciplinary program team, they can support other programs in both their inter- and transdisciplinary research endeavors.

Julie Thompson Klein’s contributions to interdisciplinary and transdisciplinary research have enriched the way collaboration is discussed and handled by introducing concepts of boundary work and boundary crossing from the field of Science and Technology Studies. In recent years, she has been integrating those concepts into crossdisciplinarity, an effort culminating in the development of a framework for a forthcoming book (Beyond Interdisciplinarity: Boundary Work, Collaboration, and Communication in the 21st Century). With her permission, we have used an earlier version of her framework to analyze boundary work and boundary crossing in transdisciplinary sustainable water management projects in Australia and Switzerland. The aim of using the framework has been twofold: to explore and assess the heuristic value of the framework, i.e. how it improves our conceptualization of boundary work in the two projects, and to examine the framework itself, i.e. whether some of the seven concepts involved are hard to work with or should be further developed.

Recent empirical studies show a persistent gap between 'socially robust' knowledge produced by transdisciplinary research projects and its ability to promote change on a large scale. Current discourses about the 'project-to-science-and-practice-at-large gap' have focused mainly on exploring various conditions that need to be fulfilled to produce 'socially robust' knowledge. Yet, those discourses have rarely built on the broader literature of knowledge utilization, which Greenhalgh and Wieringa (2011) emphasize acknowledges 'the fundamentally social ways in which knowledge emerges, circulates, and gets applied in practice.' Their insights are helpful in advancing our understanding of why transdisciplinary research projects do or do not contribute to sustainability on a large scale. Expanding Jahn et al. (2012)'s model of transdisciplinary research, we present a revised conceptual model of an ideal-typical, interactive and iterative transdisciplinary research process that adds two new phases from the field of knowledge utilization to their original three-phase model and accounts for the social and relational nature of knowledge utilization. The revised model includes five phases through which transdisciplinary projects operate in different order: (i) defining sustainability problems, (ii) producing new knowledge, (iii) assessing new knowledge, (iv) disseminating new knowledge in realms of both science and practice and (v) using new knowledge in both realms.

Research and policy processes in many fields, such as sustainability and health, are increasingly relying on transdisciplinary cooperation among a multitude of governmental, nongovernmental, and private actors from local to global levels. In the absence of hierarchical chains of command, multistakeholder governance may accommodate conflicting or diverse interests and facilitate collective action, but its effectiveness depends on its capacity to integrate systems, transformation, and target knowledge. Approaches to foster such governance are nascent and quickly evolving, and methodological standards to facilitate comparison and learning from best practice are needed. However, there is currently no evaluation approach that (i) comprehensively assesses the capacity for knowledge integration in multistakeholder governance, (ii) draws on the best available knowledge that is being developed in various fields, and (iii) combines a systematic and transferable methodological design with pragmatic feasibility.

We brought together 20 experts from institutions in nine countries, all working on evaluation approaches for collaborative science–policy initiatives. In a synthesis process that included a 2-day workshop and follow-up work among a core group of participants, we developed a tool for evaluating knowledge integration capacity in multistakeholder governance (EVOLvINC). Its 23 indicators incorporate previously defined criteria and components of transdisciplinary evaluations into a single, comprehensive framework that operationalizes the capacity for integrating systems, target, and transformation knowledge during an initiative’s (a) design and planning processes at the policy formulation stage, (b) organization and working processes at the implementation stage, and (c) sharing and learning processes at the evaluation stage of the policy cycle. EVOLvINC is (i) implemented through a questionnaire, (ii) builds on established indicators where possible, (iii) offers a consistent and transparent semiquantitative scoring and aggregation algorithm, and (iv) uses spider diagrams for visualizing results. The tool builds on experience and expertise from both the northern and southern hemispheres and was empirically validated with seven science–policy initiatives in six African and Asian countries.

As a generalized framework, EVOLvINC thus enables a structured reflection on the capacity of multistakeholder governance processes to foster knowledge integration. Its emphasis on dialog and exploration allows adaptation to contextual specificities, highlights relative strengths and weaknesses, and suggests avenues for shaping multistakeholder governance toward mutual learning, capacity building, and strengthened networks. The validation suggests that the adaptive capacity of multistakeholder governance could be best enhanced by considering systems characteristics at the policy formulation stage and fostering adaptive and generic learning at the evaluation stage of the policy cycle.

An increasing number of knowledge brokers work at the interface between research, policy and practice. Their function is to facilitate processes to foster mutual learning among research, policy and practice. For some knowledge brokers, practical methodologies to assess the quality of their work is an important concern. While frameworks exist for assessing research impact at the level of a project or program, few are available for assessing contributions of individual knowledge brokers. In response to this, we have compiled a set of indicators to measure the quantity and quality of the contributions of individual knowledge brokers to projects, programs or platforms at the interface between research, policy and practice. The set is based on a review of the literature and the experience of a group of knowledge brokers active in water research and management in Switzerland, including the co-authors of this article. The set can be used by knowledge brokers to identify ways to improve the effectiveness of their practices and to demonstrate the benefit of their work to their employers and other stakeholders. Our approach is flexible enough that it can be applied where there are limited resources available for assessment.