Department Environmental Social Sciences

Serious Games as an Element of Mulit-Criteria Decision Analysis

Structured methods are needed to support environmental policy decisions, including in the water sector. Multi-criteria decision analysis – or MCDA – is one of them, aiming at finding consensus between stakeholders with differing opinions. Traditionally, stakeholders are asked during interviews about their preferences concerning the outcomes of a set of decision options. Particularly, they have to give valuations by assigning weights to different objectives which are important in this decision. This is a tricky job as overall consequences of a weight are hard to understand, as people readily run into systematic biases during preference formulation processes, and as the interviewer might influence the interviewee when informing about such choices.

The project aims at introducing serious games, i.e. games including a non-entertaining purpose, for this phase when stakeholders state their preferences in the decision making process. 

The study consists of two parts:

  • Can serious games as an element of the MCDA process facilitate the elicitation of preferences, and can visualization of results as part of the game be used as a consistency check? 
  • Can serious games in MCDA enable involvement of a larger public, both numerically and in terms of diversity? Would that be beneficial for reaching a consensus decision?

A literature review on serious games used in environmental management will allow to identify game characteristics that would be needed to serve our purpose: the formulation of preferences and valuation processes in decision making and the involvement of a larger public. 

Based on the review, a game – or set of games – will be tested in applied case studies, allowing evaluation of the game characteristics with regards to our aims. This will be done by comparing outcomes of different approaches (with games VS without games). The players behavior will also be a focus, aiming to observe if there is a preference typology. Pros and cons of implementing games in MCDA should be clearly stated to consider future adaptation of MCDA.

Team

PD Dr. Judit Lienert Group Leader, Group: DA Tel. +41 58 765 5574 Send Mail

Dr. Alice Aubert

ZHAW Life Sciences und Facility Management
Institute for Natural Resource Sciences                                                                                                                             

Publikationen

Extbase Variable Dump
array(2 items)
   publications => '24490,21001,17651,16797' (23 chars)
   libraryUrl => 'Recommendations for online elicitation of swing weights from citizens in env
      ironmental decision-making' (102 chars)
Extbase Variable Dump
array(4 items)
   0 => Snowflake\Publications\Domain\Model\Publicationprototypepersistent entity (uid=24490, pid=124)
      originalId => protected24490 (integer)
      authors => protected'Haag, F.; Aubert, A. H.; Lienert, J.' (56 chars)
      title => protected'ValueDecisions, a web app to support decisions with conflicting objectives, 
         multiple stakeholders, and uncertainty' (114 chars)
      journal => protected'Environmental Modelling and Software' (36 chars)
      year => protected2022 (integer)
      volume => protected150 (integer)
      issue => protected'' (0 chars)
      startpage => protected'105361 (19 pp.)' (15 chars)
      otherpage => protected'' (0 chars)
      categories => protected'multi criteria decision analysis; MCDA software; multi-attribute value theor
         y; environmental decision analysis; open source; population survey' (142 chars)
      description => protected'Complex environmental and public policy decisions profit from structured pro
         cedures such as multi-criteria decision analysis (MCDA). To support such dec
         isions, the new open source application ValueDecisions provides advanced ana
         lysis and visualization with no programming expected from users. Based on mu
         lti-attribute value theory (MAVT), it offers analysis for decisions with con
         flicting and interacting objectives, multiple stakeholders, and uncertain co
         nsequences of options. Programmed in R, the shiny web framework makes it acc
         essible via a graphical user interface in the browser. We exemplify using Va
         lueDecisions for a wastewater infrastructure planning case in the Paris regi
         on. We surveyed preferences of 655 citizens and conducted sensitivity analys
         is of preference parameters. The best management options were robust across 
         a range of preference profiles and assumptions. To evaluate the app, we deve
         loped a novel usability test based on the ISO standard for software quality 
         and surveyed students using ValueDecisions for case studies.' (1048 chars)
      serialnumber => protected'1364-8152' (9 chars)
      doi => protected'10.1016/j.envsoft.2022.105361' (29 chars)
      uid => protected24490 (integer)
      _localizedUid => protected24490 (integer)modified
      _languageUid => protectedNULL
      _versionedUid => protected24490 (integer)modified
      pid => protected124 (integer)
   1 => Snowflake\Publications\Domain\Model\Publicationprototypepersistent entity (uid=21001, pid=124)
      originalId => protected21001 (integer)
      authors => protected'Aubert, A. H.; Esculier, F.; Lienert, J.' (60 chars)
      title => protected'Recommendations for online elicitation of swing weights from citizens in env
         ironmental decision-making' (102 chars)
      journal => protected'Operations Research Perspectives' (32 chars)
      year => protected2020 (integer)
      volume => protected7 (integer)
      issue => protected'' (0 chars)
      startpage => protected'100156 (13 pp.)' (15 chars)
      otherpage => protected'' (0 chars)
      categories => protected'Behavioral Operational Research; OR in environment and climate change; learn
         ing; Multi-Criteria Decision Analysis; Decision Support System; public parti
         cipation; e-democracy' (173 chars)
      description => protected'There is a growing demand for public participation in environmental decision
         -making. However, it is unclear how a large number of citizens can best enga
         ge in such complex public policy decision processes. This need from the civi
         l society challenges the OR community to develop online decision-making tool
         s. This article reports on a feasibility assessment of swing weight elicitat
         ion, implemented online, for real-world decisions about future wastewater in
         frastructure. Eliciting weights with the swing method is common in MAVT/MAUT
         , but not online. A total of 298 affected citizens from the Paris region ans
         wered the online swing weight elicitation survey. Another 357 citizens direc
         tly rated objectives. Three aspects of learning in the context of MCDA were 
         considered: did participants learn facts about the wastewater topic? Did the
         y comply with the swing elicitation process, i.e. follow the instructions? D
         id participants learn about their preferences? Factual learning was limited.
          Process compliance was really low (12%), leading to a number of recommendat
         ions for improving the interface for online swing weight elicitation. The co
         llected preferences differed statistically significantly between the complia
         nt and non-compliant participants, and also between the non-compliant and di
         rect rating respondents. This emphasised the effect of the elicitation metho
         d on preference construction. Moreover, more participants experienced a stre
         ngthening of pre-existing opinions than a change in opinion, and most report
         ed being uncertain about their answers. This calls for better understanding 
         process learning and preference construction. We discuss our developed proce
         dure for online swing weight elicitation, recommend ways to improve swing on
         line surveys, and suggest interesting future research lines that would allow
          empirically verifying our propositions.' (1864 chars)
      serialnumber => protected'2214-7160' (9 chars)
      doi => protected'10.1016/j.orp.2020.100156' (25 chars)
      uid => protected21001 (integer)
      _localizedUid => protected21001 (integer)modified
      _languageUid => protectedNULL
      _versionedUid => protected21001 (integer)modified
      pid => protected124 (integer)
   2 => Snowflake\Publications\Domain\Model\Publicationprototypepersistent entity (uid=17651, pid=124)
      originalId => protected17651 (integer)
      authors => protected'Aubert, A. H.; Lienert, J.' (41 chars)
      title => protected'Gamified online survey to elicit citizens' preferences and enhance learning 
         for environmental decisions' (103 chars)
      journal => protected'Environmental Modelling and Software' (36 chars)
      year => protected2019 (integer)
      volume => protected111 (integer)
      issue => protected'' (0 chars)
      startpage => protected'1' (1 chars)
      otherpage => protected'12' (2 chars)
      categories => protected'' (0 chars)
      description => protected'Multi-Criteria Decision Analysis (MCDA) requires a critical step, namely to 
         elicit individual preferences. On the basis of learning theories, we formali
         ze preference construction as learning about facts and values, and as a proc
         ess; we also conceptualize an online preference elicitation survey that offe
         rs learning loops to increase factual learning and support preference constr
         uction. Another originality is gamification. Game elements (a narrative and 
         non-player characters as motivational affordance) keep respondents engaged i
         n the demanding task of weight elicitation. Our tool enables broad public pa
         rticipation in MCDA, allowing reliable online preference elicitation. The su
         rvey concept was tested with 107 students and a control treatment. Quantitat
         ive and qualitative data indicate that the concept works. Participants’ fa
         ctual knowledge increased. The survey helped students to learn about their o
         wn preferences concerning the importance of objectives. The practical implic
         ation is that weighting can be reliably elicited by online surveys. Particip
         ants reported a positive experience; further ways to improve it are thorough
         ly discussed.' (1153 chars)
      serialnumber => protected'1364-8152' (9 chars)
      doi => protected'10.1016/j.envsoft.2018.09.013' (29 chars)
      uid => protected17651 (integer)
      _localizedUid => protected17651 (integer)modified
      _languageUid => protectedNULL
      _versionedUid => protected17651 (integer)modified
      pid => protected124 (integer)
   3 => Snowflake\Publications\Domain\Model\Publicationprototypepersistent entity (uid=16797, pid=124)
      originalId => protected16797 (integer)
      authors => protected'Aubert, A. H.; Bauer, R.; Lienert, J.' (57 chars)
      title => protected'A review of water-related serious games to specify use in environmental Mult
         i-Criteria Decision Analysis' (104 chars)
      journal => protected'Environmental Modelling and Software' (36 chars)
      year => protected2018 (integer)
      volume => protected105 (integer)
      issue => protected'' (0 chars)
      startpage => protected'64' (2 chars)
      otherpage => protected'78' (2 chars)
      categories => protected'Multi-Criteria Decision Analysis; sustainability; serious game; gamification
         ; stakeholder participation; behavioral operational research' (136 chars)
      description => protected'Serious games and gamification are nowadays pervasive. They are used to comm
         unicate about science and sometimes to involve citizens in science (e.g. cit
         izen science). Concurrently, environmental decision analysis is challenged b
         y the high cognitive load of the decision-making process and the possible bi
         ases threatening the rationality assumptions. Difficult decision-making proc
         esses can result in incomplete preference construction, and are generally li
         mited to few participants. We reviewed 43 serious games and gamified applica
         tions related to water. We covered the broad diversity of serious games, whi
         ch could be explained by the still unsettled terminology in the research are
         a of gamification and serious gaming. We discuss how existing games could be
         nefit early steps of Multi-Criteria Decision Analysis (MCDA), including prob
         lem structuring, stakeholder analysis, defining objectives, and exploring al
         ternatives. We argue that no existing game allows for preference elicitation
         ; one of the most challenging steps of MCDA. We propose many research opport
         unities for behavioral operational research.' (1108 chars)
      serialnumber => protected'1364-8152' (9 chars)
      doi => protected'10.1016/j.envsoft.2018.03.023' (29 chars)
      uid => protected16797 (integer)
      _localizedUid => protected16797 (integer)modified
      _languageUid => protectedNULL
      _versionedUid => protected16797 (integer)modified
      pid => protected124 (integer)

ValueDecisions, a web app to support decisions with conflicting objectives, multiple stakeholders, and uncertainty

Complex environmental and public policy decisions profit from structured procedures such as multi-criteria decision analysis (MCDA). To support such decisions, the new open source application ValueDecisions provides advanced analysis and visualization with no programming expected from users. Based on multi-attribute value theory (MAVT), it offers analysis for decisions with conflicting and interacting objectives, multiple stakeholders, and uncertain consequences of options. Programmed in R, the shiny web framework makes it accessible via a graphical user interface in the browser. We exemplify using ValueDecisions for a wastewater infrastructure planning case in the Paris region. We surveyed preferences of 655 citizens and conducted sensitivity analysis of preference parameters. The best management options were robust across a range of preference profiles and assumptions. To evaluate the app, we developed a novel usability test based on the ISO standard for software quality and surveyed students using ValueDecisions for case studies.
Haag, F.; Aubert, A. H.; Lienert, J. (2022) ValueDecisions, a web app to support decisions with conflicting objectives, multiple stakeholders, and uncertainty, Environmental Modelling and Software, 150, 105361 (19 pp.), doi:10.1016/j.envsoft.2022.105361, Institutional Repository

Recommendations for online elicitation of swing weights from citizens in environmental decision-making

There is a growing demand for public participation in environmental decision-making. However, it is unclear how a large number of citizens can best engage in such complex public policy decision processes. This need from the civil society challenges the OR community to develop online decision-making tools. This article reports on a feasibility assessment of swing weight elicitation, implemented online, for real-world decisions about future wastewater infrastructure. Eliciting weights with the swing method is common in MAVT/MAUT, but not online. A total of 298 affected citizens from the Paris region answered the online swing weight elicitation survey. Another 357 citizens directly rated objectives. Three aspects of learning in the context of MCDA were considered: did participants learn facts about the wastewater topic? Did they comply with the swing elicitation process, i.e. follow the instructions? Did participants learn about their preferences? Factual learning was limited. Process compliance was really low (12%), leading to a number of recommendations for improving the interface for online swing weight elicitation. The collected preferences differed statistically significantly between the compliant and non-compliant participants, and also between the non-compliant and direct rating respondents. This emphasised the effect of the elicitation method on preference construction. Moreover, more participants experienced a strengthening of pre-existing opinions than a change in opinion, and most reported being uncertain about their answers. This calls for better understanding process learning and preference construction. We discuss our developed procedure for online swing weight elicitation, recommend ways to improve swing online surveys, and suggest interesting future research lines that would allow empirically verifying our propositions.
Aubert, A. H.; Esculier, F.; Lienert, J. (2020) Recommendations for online elicitation of swing weights from citizens in environmental decision-making, Operations Research Perspectives, 7, 100156 (13 pp.), doi:10.1016/j.orp.2020.100156, Institutional Repository

Gamified online survey to elicit citizens' preferences and enhance learning for environmental decisions

Multi-Criteria Decision Analysis (MCDA) requires a critical step, namely to elicit individual preferences. On the basis of learning theories, we formalize preference construction as learning about facts and values, and as a process; we also conceptualize an online preference elicitation survey that offers learning loops to increase factual learning and support preference construction. Another originality is gamification. Game elements (a narrative and non-player characters as motivational affordance) keep respondents engaged in the demanding task of weight elicitation. Our tool enables broad public participation in MCDA, allowing reliable online preference elicitation. The survey concept was tested with 107 students and a control treatment. Quantitative and qualitative data indicate that the concept works. Participants’ factual knowledge increased. The survey helped students to learn about their own preferences concerning the importance of objectives. The practical implication is that weighting can be reliably elicited by online surveys. Participants reported a positive experience; further ways to improve it are thoroughly discussed.
Aubert, A. H.; Lienert, J. (2019) Gamified online survey to elicit citizens' preferences and enhance learning for environmental decisions, Environmental Modelling and Software, 111, 1-12, doi:10.1016/j.envsoft.2018.09.013, Institutional Repository

A review of water-related serious games to specify use in environmental Multi-Criteria Decision Analysis

Serious games and gamification are nowadays pervasive. They are used to communicate about science and sometimes to involve citizens in science (e.g. citizen science). Concurrently, environmental decision analysis is challenged by the high cognitive load of the decision-making process and the possible biases threatening the rationality assumptions. Difficult decision-making processes can result in incomplete preference construction, and are generally limited to few participants. We reviewed 43 serious games and gamified applications related to water. We covered the broad diversity of serious games, which could be explained by the still unsettled terminology in the research area of gamification and serious gaming. We discuss how existing games could benefit early steps of Multi-Criteria Decision Analysis (MCDA), including problem structuring, stakeholder analysis, defining objectives, and exploring alternatives. We argue that no existing game allows for preference elicitation; one of the most challenging steps of MCDA. We propose many research opportunities for behavioral operational research.
Aubert, A. H.; Bauer, R.; Lienert, J. (2018) A review of water-related serious games to specify use in environmental Multi-Criteria Decision Analysis, Environmental Modelling and Software, 105, 64-78, doi:10.1016/j.envsoft.2018.03.023, Institutional Repository
To the library

Contact

PD Dr. Judit Lienert Group Leader, Group: DA Tel. +41 58 765 5574 Send Mail

Information

Project start: October 2015

Projekt duration: 2 Jahre

Projekt type: Eawag postdoc positions