We test a big data workflow for understanding and predicting plankton dynamics using monitoring data. This is a timely case study, given the current streaming of plankton images from automated monitoring, which need to be classified and analysed. Automated classification of plankton pictures is challenging , due to the scarcity of labeled data for training classifiers and the uneven frequency of detection among species. We propose a new dataset construction and classification strategy based on semi-supervised, active, and ensemble learning. The outcoming classifiers are be used to create a large database of taxa abundances, their features, and environmental conditions as a function of time. With this dataset, we test a novel data- and modeling-oriented approach to cope with measurement choices, in such a way that the modeling potential of the dataset is maximized. The whole workflow of the proposal, from dataset construction and classification to data-oriented sampling, is explicitly devised in order to be applicable to different kinds of environmental studies.
array(3 items)0 => Snowflake\Publications\Domain\Model\Publicationprototypepersistent entity (uid=26008, pid=124)originalId => protected26008 (integer)
authors => protected'Kyathanahally, S. P.; Hardeman, T.; Reyes, M.; Merz,&nbs p;E.; Bulas, T.; Brun, P.; Pomati, F.; Baity-Jesi, M.' (149 chars)
title => protected'Ensembles of data-efficient vision transformers as a new paradigm for automa ted classification in ecology' (105 chars)
journal => protected'Scientific Reports' (18 chars)
year => protected2022 (integer)
volume => protected12 (integer)
issue => protected'' (0 chars)
startpage => protected'18590 (11 pp.)' (14 chars)
otherpage => protected'' (0 chars)
categories => protected'' (0 chars)
description => protected'Monitoring biodiversity is paramount to manage and protect natural resources . Collecting images of organisms over large temporal or spatial scales is a promising practice to monitor the biodiversity of natural ecosystems, provid ing large amounts of data with minimal interference with the environment. De ep learning models are currently used to automate classification of organism s into taxonomic units. However, imprecision in these classifiers introduces a measurement noise that is difficult to control and can significantly hind er the analysis and interpretation of data. We overcome this limitation thro ugh ensembles of Data-efficient image Transformers (DeiTs), which not only a re easy to train and implement, but also significantly outperform the previo us state of the art (SOTA). We validate our results on ten ecological imagin g datasets of diverse origin, ranging from plankton to birds. On all the dat asets, we achieve a new SOTA, with a reduction of the error with respect to the previous SOTA ranging from 29.35% to 100.00%, and often achieving perfor mances very close to perfect classification. Ensembles of DeiTs perform bett er not because of superior single-model performances but rather due to small er overlaps in the predictions by independent models and lower top-1 probabi lities. This increases the benefit of ensembling, especially when using geom etric averages to combine individual learners. While we only test our approa ch on biodiversity image datasets, our approach is generic and can be applie d to any kind of images.' (1544 chars)
serialnumber => protected'2045-2322' (9 chars)
doi => protected'10.1038/s41598-022-21910-0' (26 chars)
uid => protected26008 (integer)
_localizedUid => protected26008 (integer)modified_languageUid => protectedNULL
_versionedUid => protected26008 (integer)modifiedpid => protected124 (integer)1 => Snowflake\Publications\Domain\Model\Publicationprototypepersistent entity (uid=24021, pid=124)originalId => protected24021 (integer)
authors => protected'Kyathanahally, S. P.; Hardeman, T.; Merz, E.; Bulas,&nbs p;T.; Reyes, M.; Isles, P.; Pomati, F.; Baity-Jesi, M.' (150 chars)
title => protected'Deep learning classification of Lake Zooplankton' (48 chars)
journal => protected'Frontiers in Microbiology' (25 chars)
year => protected2021 (integer)
volume => protected12 (integer)
issue => protected'' (0 chars)
startpage => protected'746297 (13 pp.)' (15 chars)
otherpage => protected'' (0 chars)
categories => protected'plankton camera; deep learning; plankton classification; transfer learning; Greifensee; ensemble learning; fresh water; lake plankton images' (140 chars)
description => protected'Plankton are effective indicators of environmental change and ecosystem heal th in freshwater habitats, but collection of plankton data using manual micr oscopic methods is extremely labor-intensive and expensive. Automated plankt on imaging offers a promising way forward to monitor plankton communities wi th high frequency and accuracy in real-time. Yet, manual annotation of milli ons of images proposes a serious challenge to taxonomists. Deep learning cla ssifiers have been successfully applied in various fields and provided encou raging results when used to categorize marine plankton images. Here, we pres ent a set of deep learning models developed for the identification of lake p lankton, and study several strategies to obtain optimal performances, which lead to operational prescriptions for users. To this aim, we annotated into 35 classes over 17900 images of zooplankton and large phytoplankton colonies , detected in Lake Greifensee (Switzerland) with the Dual Scripps Plankton C amera. Our best models were based on transfer learning and ensembling, which classified plankton images with 98% accuracy and 93% F1 score. When tested on freely available plankton datasets produced by other automated imaging to ols (ZooScan, Imaging FlowCytobot, and ISIIS), our models performed better t han previously used models. Our annotated data, code and classification mode ls are freely available online.' (1399 chars)
serialnumber => protected'' (0 chars)
doi => protected'10.3389/fmicb.2021.746297' (25 chars)
uid => protected24021 (integer)
_localizedUid => protected24021 (integer)modified_languageUid => protectedNULL
_versionedUid => protected24021 (integer)modifiedpid => protected124 (integer)2 => Snowflake\Publications\Domain\Model\Publicationprototypepersistent entity (uid=23438, pid=124)originalId => protected23438 (integer)
authors => protected'Merz, E.; Kozakiewicz, T.; Reyes, M.; Ebi, C.; Isles,&nb sp;P.; Baity-Jesi, M.; Roberts, P.; Jaffe, J. S.; Dennis , S. R.; Hardeman, T.; Stevens, N.; Lorimer, T.; Po mati, F.' (241 chars)
title => protected'Underwater dual-magnification imaging for automated lake plankton monitoring' (76 chars)
journal => protected'Water Research' (14 chars)
year => protected2021 (integer)
volume => protected203 (integer)
issue => protected'' (0 chars)
startpage => protected'117524 (12 pp.)' (15 chars)
otherpage => protected'' (0 chars)
categories => protected'phytoplankton; zooplankton; size; microscopy; classification; diversity' (71 chars)
description => protected'The Dual Scripps Plankton Camera (DSPC) is a new approach for automated in-s itu monitoring of phyto- and zooplankton communities based on a dual magnifi cation dark-field imaging microscope. Here, we present the DSPC and its asso ciated image processing while evaluating its capabilities in i) detecting an d characterizing plankton species of different size and taxonomic categories and ii) measuring their abundance in both laboratory and field applications . In the laboratory, body size and abundance estimates by the DSPC significa ntly and robustly scaled with measurements derived by microscopy. In the fie ld, a DSPC installed permanently at 3 m depth in Lake Greifensee (Switzerlan d) delivered images of plankton individuals, colonies, and heterospecific ag gregates at hourly timescales without disrupting natural arrangements of int eracting organisms, their microenvironment or their behavior. The DSPC was a ble to track the dynamics of taxa, mostly at the genus level, in the size ra nge between ∼10 μm to ∼ 1 cm, covering many components of the planktoni c food web (including parasites and potentially toxic cyanobacteria). Compar ing data from the field-deployed DSPC to traditional sampling and microscopy revealed a general overall agreement in estimates of plankton diversity and abundances. The most significant disagreements between traditional methods and the DSPC resided in the measurements of zooplankton community properties . Our data suggest that the DSPC is better equipped to study the dynamics an d demography of heterogeneously distributed organisms such as zooplankton, b ecause high temporal resolution and continuous sampling offer more informati on and less variability in taxa detection and quantification than traditiona l sampling. Time series collected by the DSPC depicted ecological succession patterns, algal bloom dynamics and diel fluctuations with a temporal freque ncy and morphological resolution that was never observed by traditional meth ods. Access to high freq...' (2287 chars)
serialnumber => protected'0043-1354' (9 chars)
doi => protected'10.1016/j.watres.2021.117524' (28 chars)
uid => protected23438 (integer)
_localizedUid => protected23438 (integer)modified_languageUid => protectedNULL
_versionedUid => protected23438 (integer)modifiedpid => protected124 (integer)
Ensembles of data-efficient vision transformers as a new paradigm for automated classification in ecology
Monitoring biodiversity is paramount to manage and protect natural resources. Collecting images of organisms over large temporal or spatial scales is a promising practice to monitor the biodiversity of natural ecosystems, providing large amounts of data with minimal interference with the environment. Deep learning models are currently used to automate classification of organisms into taxonomic units. However, imprecision in these classifiers introduces a measurement noise that is difficult to control and can significantly hinder the analysis and interpretation of data. We overcome this limitation through ensembles of Data-efficient image Transformers (DeiTs), which not only are easy to train and implement, but also significantly outperform the previous state of the art (SOTA). We validate our results on ten ecological imaging datasets of diverse origin, ranging from plankton to birds. On all the datasets, we achieve a new SOTA, with a reduction of the error with respect to the previous SOTA ranging from 29.35% to 100.00%, and often achieving performances very close to perfect classification. Ensembles of DeiTs perform better not because of superior single-model performances but rather due to smaller overlaps in the predictions by independent models and lower top-1 probabilities. This increases the benefit of ensembling, especially when using geometric averages to combine individual learners. While we only test our approach on biodiversity image datasets, our approach is generic and can be applied to any kind of images.
Kyathanahally, S. P.; Hardeman, T.; Reyes, M.; Merz, E.; Bulas, T.; Brun, P.; Pomati, F.; Baity-Jesi, M. (2022) Ensembles of data-efficient vision transformers as a new paradigm for automated classification in ecology, Scientific Reports, 12, 18590 (11 pp.), doi:10.1038/s41598-022-21910-0, Institutional Repository
Deep learning classification of Lake Zooplankton
Plankton are effective indicators of environmental change and ecosystem health in freshwater habitats, but collection of plankton data using manual microscopic methods is extremely labor-intensive and expensive. Automated plankton imaging offers a promising way forward to monitor plankton communities with high frequency and accuracy in real-time. Yet, manual annotation of millions of images proposes a serious challenge to taxonomists. Deep learning classifiers have been successfully applied in various fields and provided encouraging results when used to categorize marine plankton images. Here, we present a set of deep learning models developed for the identification of lake plankton, and study several strategies to obtain optimal performances, which lead to operational prescriptions for users. To this aim, we annotated into 35 classes over 17900 images of zooplankton and large phytoplankton colonies, detected in Lake Greifensee (Switzerland) with the Dual Scripps Plankton Camera. Our best models were based on transfer learning and ensembling, which classified plankton images with 98% accuracy and 93% F1 score. When tested on freely available plankton datasets produced by other automated imaging tools (ZooScan, Imaging FlowCytobot, and ISIIS), our models performed better than previously used models. Our annotated data, code and classification models are freely available online.
Kyathanahally, S. P.; Hardeman, T.; Merz, E.; Bulas, T.; Reyes, M.; Isles, P.; Pomati, F.; Baity-Jesi, M. (2021) Deep learning classification of Lake Zooplankton, Frontiers in Microbiology, 12, 746297 (13 pp.), doi:10.3389/fmicb.2021.746297, Institutional Repository
Underwater dual-magnification imaging for automated lake plankton monitoring
The Dual Scripps Plankton Camera (DSPC) is a new approach for automated in-situ monitoring of phyto- and zooplankton communities based on a dual magnification dark-field imaging microscope. Here, we present the DSPC and its associated image processing while evaluating its capabilities in i) detecting and characterizing plankton species of different size and taxonomic categories and ii) measuring their abundance in both laboratory and field applications. In the laboratory, body size and abundance estimates by the DSPC significantly and robustly scaled with measurements derived by microscopy. In the field, a DSPC installed permanently at 3 m depth in Lake Greifensee (Switzerland) delivered images of plankton individuals, colonies, and heterospecific aggregates at hourly timescales without disrupting natural arrangements of interacting organisms, their microenvironment or their behavior. The DSPC was able to track the dynamics of taxa, mostly at the genus level, in the size range between ∼10 μm to ∼ 1 cm, covering many components of the planktonic food web (including parasites and potentially toxic cyanobacteria). Comparing data from the field-deployed DSPC to traditional sampling and microscopy revealed a general overall agreement in estimates of plankton diversity and abundances. The most significant disagreements between traditional methods and the DSPC resided in the measurements of zooplankton community properties. Our data suggest that the DSPC is better equipped to study the dynamics and demography of heterogeneously distributed organisms such as zooplankton, because high temporal resolution and continuous sampling offer more information and less variability in taxa detection and quantification than traditional sampling. Time series collected by the DSPC depicted ecological succession patterns, algal bloom dynamics and diel fluctuations with a temporal frequency and morphological resolution that was never observed by traditional methods. Access to high frequency, reproducible and real-time data of a large spectrum of the planktonic ecosystem expands our understanding of both applied and fundamental plankton ecology. We conclude the DSPC is robust for both research and water quality monitoring and suitable for stable long-term deployments.
Merz, E.; Kozakiewicz, T.; Reyes, M.; Ebi, C.; Isles, P.; Baity-Jesi, M.; Roberts, P.; Jaffe, J. S.; Dennis, S. R.; Hardeman, T.; Stevens, N.; Lorimer, T.; Pomati, F. (2021) Underwater dual-magnification imaging for automated lake plankton monitoring, Water Research, 203, 117524 (12 pp.), doi:10.1016/j.watres.2021.117524, Institutional Repository
