Eawag - Swiss Federal Institute of Aquatic Science and Technology

Background

Community assembly in ecosystems depends on various factors: On which species are present in the region, on local environmental conditions, and on the interactions between species. Different methods can be used to predict the occurrence probability of different species. These are either based on the mathematical formulation of existing knowledge about the mechanisms that influence species occurrence. Or they are based on statistical data analysis. In addition, there are machine learning methods, where it is more difficult to understand relations with environmental factors. All three methods have advantages and disadvantages, but have rarely been used together because they differ in their requirements for data and computational effort.

Content and aim of the research project

In this project, mechanistic models, statistical models, and machine learning methods will be applied to improve knowledge of environmental effects on macroinvertebrates in streams, so that in the future we can better predict which species occur under which environmental conditions. We will use the knowledge gained to develop a model that is as simple and efficient as possible, contains the most important mechanisms and is optimized for prediction. Another goal is to use the computer simulations to identify macroinvertebrate community metrics that are as sensitive as possible to future environmental changes.

Scientific and societal context of the research project

Our work will generate new insights for predicting community assembly that can inform the management of aquatic ecosystems in response to future environmental changes.

Extbase Variable Dump

array(2 items)
   publications => '32476,30648' (11 chars)
   libraryUrl => '' (0 chars)

Extbase Variable Dump

array(2 items)
   0 => Snowflake\Publications\Domain\Model\Publicationprototypepersistent entity (uid=32476, pid=124)
      originalId => protected32476 (integer)
      authors => protected'Khaliq,&nbsp;I.; Chollet Ramampiandra,&nbsp;E.; Vorburger,&nbsp;C.; Narwani,
         &nbsp;A.; Schuwirth,&nbsp;N.' (104 chars)
      title => protected'The effect of water temperature changes on biological water quality assessme
         nt' (78 chars)
      journal => protected'Ecological Indicators' (21 chars)
      year => protected2024 (integer)
      volume => protected159 (integer)
      issue => protected'' (0 chars)
      startpage => protected'111652 (10 pp.)' (15 chars)
      otherpage => protected'' (0 chars)
      categories => protected'biological indices; macroinvertebrate species richness; IBCH index; SPEARpes
         cticides index; climate change; water quality assessment' (132 chars)
      description => protected'Increasing temperatures caused by anthropogenic climate change are leading t
         o changes in the composition of local communities across biomes. This has im
         plications for ecological assessment methods that rely on macroinvertebrates
          as bioindicators of water quality. To investigate the influence of changing
          water temperature on these assessment methods, we analysed macroinvertebrat
         e data from Swiss national monitoring programs. We used a species distributi
         on model to simulate temperature change effects on macroinvertebrate communi
         ties and estimated the resulting changes on three biological indices commonl
         y used in Switzerland, namely the species richness of Ephemeroptera, Plecopt
         era and Trichoptera (EPT), the Swiss biological (IBCH) index along with its 
         components, as well as the species at risk pesticides (SPEAR<sub>pesticides<
         /sub>) index. While results vary by temperature scenario and index, our mode
         l results for the most realistic water temperature increase scenario of + 
         2 °C across most sites in Switzerland suggest no, or only a minor, influen
         ce of temperature (not accounting for other hydrological changes). Our model
          projection predicted only a small increase in the probability of occurrence
          for 70 % of the studied families. The sensitivity to temperature as captur
         ed in our model is generally not very high and varies among the biological i
         ndices: on average across all sites, a + 2 °C increase in temperature re
         sulted in a 7 % increase in EPT species richness, a 4 % increase in the IB
         CH index, and a less than 1 % increase in the SPEAR<sub>pesticides</sub> in
         dex. Our study suggests the robustness of these biological indices to modera
         te warming and points towards the usefulness of these biological indices for
          the next few decades as tools for water quality assessment. Despite some li
         mitations of statistical species distribution models (e.g., not accounting f
         or dispersal limitation or biotic interactions, predictive performance varyi
         ng by taxon), the study ...' (2280 chars)
      serialnumber => protected'1470-160X' (9 chars)
      doi => protected'10.1016/j.ecolind.2024.111652' (29 chars)
      uid => protected32476 (integer)
      _localizedUid => protected32476 (integer)modified
      _languageUid => protectedNULL
      _versionedUid => protected32476 (integer)modified
      pid => protected124 (integer)
   1 => Snowflake\Publications\Domain\Model\Publicationprototypepersistent entity (uid=30648, pid=124)
      originalId => protected30648 (integer)
      authors => protected'Chollet Ramampiandra,&nbsp;E.; Scheidegger,&nbsp;A.; Wydler,&nbsp;J.; Schuwi
         rth,&nbsp;N.' (88 chars)
      title => protected'A comparison of machine learning and statistical species distribution models
         : Quantifying overfitting supports model interpretation' (131 chars)
      journal => protected'Ecological Modelling' (20 chars)
      year => protected2023 (integer)
      volume => protected481 (integer)
      issue => protected'' (0 chars)
      startpage => protected'110353 (11 pp.)' (15 chars)
      otherpage => protected'' (0 chars)
      categories => protected'species distribution model; statistical models; interpretable machine learni
         ng; model complexity; freshwater macroinvertebrates' (127 chars)
      description => protected'Species distribution models are commonly applied to predict species response
         s to environmental conditions. A wide variety of models with different prope
         rties exist that vary in complexity, which affects their predictive performa
         nce and interpretability. Machine learning algorithms are increasingly used 
         because they are capable to capture complex relationships and are often bett
         er in prediction. However, to inform environmental management, it is importa
         nt that a model predicts well for the right reasons. It remains a challenge 
         to select a model with a reasonable level of complexity that captures the tr
         ue relationship between the response and explanatory variables as good as po
         ssible rather than fitting to the noise in the data.<br />In this study we a
         sk: 1) how much predictive performance can we gain by using increasingly com
         plex models, 2) how does model complexity affect the degree of overfitting, 
         and 3) do the inferred responses differ among models and what can we learn f
         rom them? To address these questions, we applied eight models with different
          complexity to predict the probability of occurrence of freshwater macroinve
         rtebrate taxa based on 2729 Swiss monitoring samples. We compared the models
          in terms of predictive performance during cross-validation and for generali
         zation out of the calibration domain ("extrapolation" or transferability). W
         e applied model agnostic tools to shed light on model interpretability.<br /
         >Contrary to our expectation, all models predicted similarly well during cro
         ss-validation, while no model predicted better than the null model during ou
         t-of-domain generalization on average over all taxa. Performance was best fo
         r taxa with intermediate prevalence. More complex models predicted slightly 
         better than standard statistical models but were prone to overfitting.<br />
         Overfitting indicates that a model describes not only the signal in the data
          but also part of the noise. This impedes the interpretation of response sha
         pes learned by the model...' (2835 chars)
      serialnumber => protected'0304-3800' (9 chars)
      doi => protected'10.1016/j.ecolmodel.2023.110353' (31 chars)
      uid => protected30648 (integer)
      _localizedUid => protected30648 (integer)modified
      _languageUid => protectedNULL
      _versionedUid => protected30648 (integer)modified
      pid => protected124 (integer)

Increasing temperatures caused by anthropogenic climate change are leading to changes in the composition of local communities across biomes. This has implications for ecological assessment methods that rely on macroinvertebrates as bioindicators of water quality. To investigate the influence of changing water temperature on these assessment methods, we analysed macroinvertebrate data from Swiss national monitoring programs. We used a species distribution model to simulate temperature change effects on macroinvertebrate communities and estimated the resulting changes on three biological indices commonly used in Switzerland, namely the species richness of Ephemeroptera, Plecoptera and Trichoptera (EPT), the Swiss biological (IBCH) index along with its components, as well as the species at risk pesticides (SPEAR_pesticides) index. While results vary by temperature scenario and index, our model results for the most realistic water temperature increase scenario of + 2 °C across most sites in Switzerland suggest no, or only a minor, influence of temperature (not accounting for other hydrological changes). Our model projection predicted only a small increase in the probability of occurrence for 70 % of the studied families. The sensitivity to temperature as captured in our model is generally not very high and varies among the biological indices: on average across all sites, a + 2 °C increase in temperature resulted in a 7 % increase in EPT species richness, a 4 % increase in the IBCH index, and a less than 1 % increase in the SPEAR_pesticides index. Our study suggests the robustness of these biological indices to moderate warming and points towards the usefulness of these biological indices for the next few decades as tools for water quality assessment. Despite some limitations of statistical species distribution models (e.g., not accounting for dispersal limitation or biotic interactions, predictive performance varying by taxon), the study provides valuable insights into the complex relationships between environmental factors and macroinvertebrate communities, and the potential impacts of future temperature change. These findings can inform conservation and management efforts for these important ecological systems.

Khaliq, I.; Chollet Ramampiandra, E.; Vorburger, C.; Narwani, A.; Schuwirth, N. (2024) The effect of water temperature changes on biological water quality assessment, Ecological Indicators, 159, 111652 (10 pp.), doi:10.1016/j.ecolind.2024.111652, Institutional Repository

Species distribution models are commonly applied to predict species responses to environmental conditions. A wide variety of models with different properties exist that vary in complexity, which affects their predictive performance and interpretability. Machine learning algorithms are increasingly used because they are capable to capture complex relationships and are often better in prediction. However, to inform environmental management, it is important that a model predicts well for the right reasons. It remains a challenge to select a model with a reasonable level of complexity that captures the true relationship between the response and explanatory variables as good as possible rather than fitting to the noise in the data.
In this study we ask: 1) how much predictive performance can we gain by using increasingly complex models, 2) how does model complexity affect the degree of overfitting, and 3) do the inferred responses differ among models and what can we learn from them? To address these questions, we applied eight models with different complexity to predict the probability of occurrence of freshwater macroinvertebrate taxa based on 2729 Swiss monitoring samples. We compared the models in terms of predictive performance during cross-validation and for generalization out of the calibration domain ("extrapolation" or transferability). We applied model agnostic tools to shed light on model interpretability.
Contrary to our expectation, all models predicted similarly well during cross-validation, while no model predicted better than the null model during out-of-domain generalization on average over all taxa. Performance was best for taxa with intermediate prevalence. More complex models predicted slightly better than standard statistical models but were prone to overfitting.
Overfitting indicates that a model describes not only the signal in the data but also part of the noise. This impedes the interpretation of response shapes learned by the model, because one cannot distinguish the signal from the noise. Furthermore, the strongly overfitting models learned irregular relationships and strong interactions that are ecologically not plausible. Thus, in this study, the minor gain in predictive performance from more complex models was outweighed by the overfitting.
Ecological field data that is used as model input or for calibration is typically prone to different sources of variability, from sampling, the measurement process and stochasticity. We therefore call for caution when using complex data-driven models to learn about species responses or to inform environmental management. In such cases, we recommend to compare a range of models regarding their predictive performance, overfitting and response shapes to better understand the robustness of inferred responses.

Chollet Ramampiandra, E.; Scheidegger, A.; Wydler, J.; Schuwirth, N. (2023) A comparison of machine learning and statistical species distribution models: Quantifying overfitting supports model interpretation, Ecological Modelling, 481, 110353 (11 pp.), doi:10.1016/j.ecolmodel.2023.110353, Institutional Repository

Dr. Nele Schuwirth Head of Department and Group Leader (she/her) Tel. +41 58 765 5528 Send Mail

Andreas Scheidegger Statistics, Data Science & Modeling Tel. +41 58 765 5053 Send Mail

Dr. Emma Chollet Ramampiandra

PhD student

Department Systems Analysis, Integrated Assessment and Modelling

Bridging the gap between data science and mechanistic modelling to gain knowledge about community assembly

Publications

Team

Contact