Neural likelihood estimation methods for simulation-based inference can suffer from performance degradation when the modeled data is very high-dimensional or lies along a lower-dimensional manifold, which is due to the inability of the density estimator to accurately estimate a density function. We present Surjective Sequential Neural Likelihood (SSNL) estimation, a novel member in the family of methods for simulation-based inference (SBI). SSNL fits a dimensionality-reducing surjective normalizing flow model and uses it as a surrogate likelihood function, which allows for computational inference via Markov chain Monte Carlo or variational Bayes methods. Among other benefits, SSNL avoids the requirement to manually craft summary statistics for inference of high-dimensional data sets, since the lower-dimensional representation is computed simultaneously with learning the likelihood and without additional computational overhead. We evaluate SSNL on a wide variety of experiments, including two challenging real-world examples from the astrophysics and neuroscience literatures, and show that it either outperforms or is on par with state-of-the-art methods, making it an excellent off-the-shelf estimator for SBI for high-dimensional data sets.

1. Particle filters and smoothers are sequential Monte Carlo algorithms used to fit non-linear, non-Gaussian state-space models. These algorithms are well placed to fit process-oriented models to animal-tracking data, especially in receiver arrays, but to date they have received limited attention in the ecological literature.
2. We introduce a Bayesian filtering–smoothing algorithm that reconstructs individual movements and patterns of space use from animal-tracking data, with a focus on passive acoustic telemetry systems. Within a sound probabilistic framework, the methodology integrates the movement process and the observation processes of disparate datasets, while correctly representing uncertainty. In a simulation-based analysis, we compare the performance of our algorithm to the prevailing heuristic methods used to study movements and space use in passive acoustic telemetry systems and analyse algorithm sensitivity.
3. We find the particle smoothing methodology outperforms heuristic methods across the board. Particle-based maps represent simulated movements more accurately, even in dense receiver arrays, and are better suited to analyses of home ranges, residency and habitat preferences.
4. This study sets a new state-of-the-art for movement modelling in receiver arrays. Particle algorithms provide a robust, flexible and intuitive modelling framework with potential applications in many ecological settings.

1. State-space models are a powerful modelling framework in movement ecology that represents individual movements and the processes connecting movements to observations. However, fitting state-space models to animal-tracking data can be difficult and computationally expensive.
2. Here, we introduce patter, a package that provides particle filtering and smoothing algorithms that fit Bayesian state-space models to tracking data, with a focus on data from aquatic animals in receiver arrays. patter is written in R, with a performant Julia backend. Package functionality supports data simulation, preparation, filtering, smoothing and mapping.
3. In two examples, we demonstrate how to implement patter to reconstruct the movements of a tagged animal in an acoustic telemetry system from acoustic detections and ancillary observations. With perfect information, the particle filter reconstructs the true (unobserved) movement path (Example One). More generally, particle algorithms represent an individual's possible location probabilistically as a weighted series of samples (‘particles’). In our illustration, we resolve an individual's (unobserved) location every 2 min during 1 month and use particles to visualise movements, map space use and quantify residency (Example Two).
4. patter facilitates robust, flexible and efficient analyses of animal-tracking data. The methods are widely applicable and enable refined analyses of space use, home ranges and residency.

We use a zero-dimensional stochastic dynamo model describing the coupled oscillatory behavior of the toroidal and poloidal components of the solar magnetic field. The model includes a time delay, arising from the assumption that the two field components are generated in spatially segregated locations, and a quenching of the α-effect for amplitudes of the toroidal magnetic field outside a range defined by a lower and an upper threshold. We apply two distinct simulation-based Bayesian inference methods to calibrate the model parameters. The first approach employs a Simulated Annealing Approximate Bayesian Computation algorithm, based on the comparison of relevant features extracted from observed data with their counterparts from simulated data. The second one leverages a sequential neural posterior density estimation method known as Automatic Posterior Transformation. The inference was carried out using two different data sets: the observed sunspot record and a recent millennial solar activity reconstruction based on ¹⁴C cosmogenic radionuclides from tree rings. The different methods and data sets produce remarkably consistent posterior distributions for the model parameters. Bayesian inference results corroborate the hypothesis presented in a previous publication that the solar dynamo might be operating close to a critical bifurcation point characterized by the coexistence of two modes akin to normal and Grand Minima regimes. We show that the calibrated dynamo model, despite its simplicity, can be used to make predictions about solar cycles.

Recent successes with machine learning (ML) models in catchment hydrology have highlighted their ability to extract crucial information from catchment properties pertinent to the rainfall–runoff relationship. In this study, we aim to identify a minimal set of catchment signatures in streamflow that, when combined with meteorological drivers, enable an accurate reconstruction of the entire streamflow time series. To achieve this, we utilize an explicit noise-conditional autoencoder (ENCA), which, assuming an optimal architecture, separates the influences of meteorological drivers and catchment properties on streamflow. The ENCA architecture feeds meteorological forcing and climate attributes into the decoder in order to incentivize the encoder to only learn features that are related to landscape properties minimally related to climate. By isolating the effect of meteorology, these hydrological features can thus be interpreted as landscape fingerprints. The optimal number of features is found by means of an intrinsic dimension estimator. We train our model on the hydro-meteorological time series data of 568 catchments of the continental United States from the Catchment Attributes and Meteorology for Large-sample Studies (CAMELS) dataset. We compare the reconstruction accuracy with models that take as input a subset of static catchment attributes (both climate and landscape attributes) along with meteorological forcing variables. Our results suggest that available landscape attributes can be summarized by only two relevant learnt features (or signatures), while at least a third one is needed for about a dozen difficult-to-predict catchments in the central United States, which is mainly characterized by a high aridity index. The principal components of the learnt features strongly correlate with the baseflow index and aridity indicators, which is consistent with the idea that these indicators capture the variability of catchment hydrological responses. The correlation analysis further indicates that soil-related and vegetation attributes are of importance.

Due to our limited knowledge about complex environmental systems, our predictions of their behavior under different scenarios or decision alternatives are subject to considerable uncertainty. As this uncertainty can often be relevant for societal decisions, the consideration, quantification and communication of it is very important. Due to internal stochasticity, often poorly known influence factors, and only partly known mechanisms, in many cases, a stochastic model is needed to get an adequate description of uncertainty. As this implies the need to infer constant parameters, as well as the time-course of stochastic model states, a very high-dimensional inference problem for model calibration has to be solved. This is very challenging from a methodological and a numerical perspective. To illustrate aspects of this problem and show options to successfully tackle it, we compare three numerical approaches: Hamiltonian Monte Carlo, Particle Markov Chain Monte Carlo, and Conditional Ornstein-Uhlenbeck Sampling. As a case study, we select the analysis of hydrological data with a stochastic hydrological model. We conclude that the performance of the investigated techniques is comparable for the analyzed system, and that also generality and practical considerations may be taken into account to guide the choice of which technique is more appropriate for a particular application.

Calibration of precipitation-streamflow models to streamflow signatures is a promising approach for streamflow prediction in ungauged basins (PUB). The estimation of parameter and prediction uncertainty in this case is not trivial because: (a) calibration takes place in the signature domain, while predictions are required in the time domain, and (b) streamflow signatures are estimated (e.g., from donor catchments) rather than “observed” (computed from observed streamflow in the target catchment), and therefore particularly uncertain. This study investigates model calibration using estimated signatures in an Approximate Bayesian Computation framework. First, we construct a stochastic signature transfer model, based on seasonal flow duration curves. Second, we calibrate a precipitation-streamflow model to the estimated signatures, accounting for their uncertainty. The proposed method is tested in six catchments of the Thur basin, Switzerland. Three data availability scenarios are considered: (a) concomitant scenario, where signatures are "observed," (b) non-concomitant scenario, where signatures are transferred from a different time period, and (c) regionalization scenario, where signatures are transferred from (neighboring) donor catchments. In this study, the switch from observed to regionalized signatures increases predictive streamflow uncertainty by 38% and worsens the (deterministic) fit to observations by 17% (in terms of Nash-Sutcliffe efficiency). Despite this deterioration, posterior predictive uncertainty remains lower than prior predictive uncertainty generated using uniform priors over representative parameter ranges (“uncalibrated” model), which demonstrates the effectiveness of the proposed signature-based calibration. More importantly, uncertainty is reliably estimated at the ungauged catchments, which represents a key advance in stochastic streamflow PUB.

Stochastic models in hydrology are very useful and widespread tools for making reliable probabilistic predictions. However, such models are only accurate at making predictions if model parameters are first of all calibrated to measured data in a consistent framework such as the Bayesian one, in which knowledge about model parameters is described through probability distributions. Unfortunately, Bayesian parameter calibration, a. k. a. inference, with stochastic models, is often a computationally intractable problem with traditional inference algorithms, such as the Metropolis algorithm, due to the expensive likelihood functions. Therefore, the prohibitive computational cost is often overcome by employing over-simplified error models, which leads to biased parameter estimates and unreliable predictions. However, thanks to recent advancements in algorithms and computing power, fully fledged Bayesian inference with stochastic models is no longer off-limits for hydrological applications. Our goal in this work is to demonstrate that a computationally efficient Hamiltonian Monte Carlo algorithm with a timescale separation makes Bayesian parameter inference with stochastic models feasible. Hydrology can potentially take great advantage of this powerful data-driven inference method as a sound calibration of model parameters is essential for making robust probabilistic predictions, which can certainly be useful in planning and policy-making. We demonstrate the Hamiltonian Monte Carlo approach by detailing a case study from urban hydrology. Discussing specific hydrological models or systems is outside the scope of our present work and will be the focus of further studies.

Deep learning methods have frequently outperformed conceptual hydrologic models in rainfall-runoff modelling. Attempts of investigating such deep learning models internally are being made, but the traceability of model states and processes and their interrelations to model input and output is not yet fully understood. Direct interpretability of mechanistic processes has always been considered an asset of conceptual models that helps to gain system understanding aside of predictability. We introduce hydrologic neural ordinary differential equation (ODE) models that perform as well as state-of-the-art deep learning methods in stream flow prediction while maintaining the ease of interpretability of conceptual hydrologic models. In neural ODEs, internal processes that are represented in differential equations, are substituted by neural networks. Therefore, neural ODE models enable the fusion of deep learning with mechanistic modelling. We demonstrate the basin-specific predictive performance for 569 catchments of the continental United States. For exemplary basins, we analyse the dynamics of states and processes learned by the model-internal neural networks. Finally, we discuss the potential of neural ODE models in hydrology.

For stochastic models with intractable likelihood functions, approximate Bayesian computation offers a way of approximating the true posterior through repeated comparisons of observations with simulated model outputs in terms of a small set of summary statistics. These statistics need to retain the information that is relevant for constraining the parameters but cancel out the noise. They can thus be seen as thermodynamic state variables, for general stochastic models. For many scientific applications, we need strictly more summary statistics than model parameters to reach a satisfactory approximation of the posterior. Therefore, we propose to use the inner dimension of deep neural network based Autoencoders as summary statistics. To create an incentive for the encoder to encode all the parameter-related information but not the noise, we give the decoder access to explicit or implicit information on the noise that has been used to generate the training data. We validate the approach empirically on two types of stochastic models.

The amplitude of the 11 yr solar cycle is well known to be subject to long-term modulation, including sustained periods of very low activity known as Grand Minima. Stable long-period cycles found in proxies of solar activity have given new momentum to the debate about a possible influence of the tiny planetary tidal forcing. Here, we study the solar cycle by means of a simple zero-dimensional dynamo model, which includes a delay caused by meridional circulation as well as a quenching of the α-effect at toroidal magnetic fields exceeding an upper threshold. Fitting this model to the sunspot record, we find a set of parameters close to the bifurcation point at which two stable oscillatory modes emerge. One mode is a limit cycle resembling normal solar activity including a characteristic kink in the decaying limb of the cycle. The other mode is a weak sub-threshold cycle that could be interpreted as Grand Minimum activity. Adding noise to the model, we show that it exhibits Stochastic Resonance, which means that a weak external modulation can toss the dynamo back and forth between these two modes, whereby the periodicities of the modulation get strongly amplified.

Key traits of unicellular species, such as cell size, often follow scale-free or self-similar distributions, hinting at the possibility of an underlying critical process. However, linking such empirical scaling laws to the critical regime of realistic individual-based model classes is difficult. Here, we reveal new empirical scaling evidence associated with a transition in the population and the chlorophyll dynamics of phytoplankton. We offer a possible explanation for these observations by deriving scaling laws in the vicinity of the critical point of a new universality class of non-local cell growth and division models. This "criticality hypothesis" can be tested through new scaling predictions derived for our model class, for the response of chlorophyll distributions to perturbations. The derived scaling laws may also be generalized to other cellular traits and environmental drivers relevant to phytoplankton ecology.

Over the past decades, advances in data collection and machine learning have paved the way for the development of autonomous simulation frameworks. Among these, many are capable not only of assimilating real‐time data to correct their predictive shortcomings but also of improving their future performance through self‐optimization. In hydrogeology, such techniques harbor great potential for informing sustainable management practices. Simulating the intricacies of groundwater flow requires an adequate representation of unknown, often highly heterogeneous geology. Unfortunately, it is difficult to reconcile the structural complexity demanded by realistic geology with the simplifying assumptions introduced in many calibration methods. The particle filter framework would provide the necessary versatility to retain such complex information but suffers from the curse of dimensionality, a fundamental limitation discouraging its use in systems with many unknowns. Due to the prevalence of such systems in hydrogeology, the particle filter has received little attention in groundwater modeling so far. In this study, we explore the combined use of dimension‐reducing techniques and artificial parameter dynamics to enable a particle filter framework for a groundwater model. Exploiting freedom in the design of the dimension‐reduction approach, we ensure consistency with a predefined geological pattern. The performance of the resulting optimizer is demonstrated in a synthetic test case for three such geological configurations and compared to two Ensemble Kalman Filter setups. Favorable results even for deliberately misspecified settings make us hopeful that nested particle filters may constitute a useful tool for geologically consistent real‐time parameter optimization.

This study investigates Bayesian signature-domain inference of hydrological models using Approximate Bayesian Computation (ABC) algorithms, and compares it to ‘‘traditional’’ time-domain inference. Our focus is on the quantification of predictive uncertainty in the streamflow time series and on understanding the information content of particular combinations of signatures. A combination of synthetic and real data experiments using conceptual rainfall-runoff models is employed. Synthetic experiments demonstrate: (i) the general consistency of signature and time-domain inferences, (ii) the ability to estimate streamflow error model parameters (reliably quantify streamflow uncertainty) even when calibrating in the signature domain, and (iii) the potential robustness of signature-domain inference when the (probabilistic) hydrological model is misspecified (e.g., by unaccounted timing errors). The experiments also suggest limitations of the signature-domain approach in terms of information loss when general (nonsufficient) statistics are used, and increased computational costs incurred by the ABC implementation. Real data experiments confirm the viability of Bayesian signature-domain inference and its general consistency with time-domain inference in terms of predictive uncertainty quantification. In addition, we demonstrate the utility of the flashiness index for the estimation of streamflow error parameters, and show that signatures based on the Flow Duration Curve alone are insufficient to calibrate parameters controlling streamflow dynamics. Overall, the study further establishes signature-domain inference (implemented using ABC) as a promising method for comparing the information content of hydrological signatures, for prediction under data-scarce conditions, and, under certain circumstances, for mitigating the impact of deficiencies in the formulation of the predictive model.

This study considers Bayesian calibration of hydrological models using streamflow signatures and its implementation using Approximate Bayesian Computation (ABC). If the modeling objective is to predict streamflow time series and associated uncertainty, a probabilistic model of streamflow must be specified but the inference equations must be developed in the signature domain. However, even starting from simple probabilistic models of streamflow time series, working in the signature domain makes the likelihood function difficult or impractical to evaluate (in particular, as it is unavailable in closed form). This challenge can be tackled using ABC, a general class of numerical algorithms for sampling from conditional distributions, such as (but not limited to) Bayesian posteriors given any calibration data. Using ABC does not avoid the requirement of Bayesian inference to specify a probability model of the data, but rather exchanges the requirement to evaluate the pdf of this model (needed to evaluate the likelihood function) by the requirement to sample model output realizations. For this reason ABC is attractive for inference in the signature domain. We clarify poorly understood aspects of ABC in the hydrological literature, including similarities and differences between ABC and GLUE, and comment on previous applications of ABC in hydrology. An error analysis of ABC approximation errors and their dependence on the tolerance is presented. An empirical case study is used to illustrate the impact of omitting the specification of a probabilistic model (and instead using a deterministic model within the ABC algorithm), and the impact of a coarse ABC tolerance.

As in many fields of dynamic modeling, the long runtime of hydrological models hinders Bayesian inference of model parameters from data. By replacing a model with an approximation of its output as a function of input and/or parameters, emulation allows us to complete this task by trading-off accuracy for speed. We combine (i) the use of a mechanistic emulator, (ii) low-discrepancy sampling of the parameter space, and (iii) iterative refinement of the design data set, to perform Bayesian inference with a very small design data set constructed with 128 model runs in a parameter space of up to eight dimensions. In our didactic example we use a model implemented with the hydrological simulator SWMM that allows us to compare our inference results against those derived with the full model. This comparison demonstrates that iterative improvements lead to reasonable results with a very small design data set.

System identification, sensitivity analysis, optimization and control, require a large number of model evaluations. Accurate simulators are too slow for these applications. Fast emulators provide a solution to this efficiency demand, sacrificing unneeded accuracy for speed. There are many strategies for developing emulators but selecting one remains subjective. Herein we compare the performance of two kinds of emulators: mechanistic emulators that use knowledge of the simulator's equations, and purely data-driven emulators using matrix factorization. We borrow simulators from urban water management, because more stringent performance criteria on water utilities have made emulation a crucial tool within this field. Results suggest that naive data-driven emulation outperforms mechanistic emulation. We discuss scenarios in which mechanistic emulation seems favorable for extrapolation in time and dealing with sparse and unevenly sampled data. We also point to advances in Machine Learning that have not permeated yet into the environmental science community.

The recent explosion of high dimensional, high resolution ‘big-data’ from automated bioinformatics measurement techniques demands new methods for unsupervised data processing. An essential analysis step is the identification of groups of similar data, or ‘clusters’, in noisy high-dimensional data spaces, as this permits to perform some analysis steps at the group level. Popular clustering algorithms introduce an undesired cluster shape bias, require prior knowledge of the number of clusters, and are unable to properly deal with noise. Manual data gating, often used to assist these methods, is based on low-dimensional projection techniques, which is prone to obscure the underlying data structure. While Hebbian Learning Clustering successfully overcomes all of these limitations (by using only local similarities to infer global structure), previous implementations were unsuited to deal with big data sets. Here, we present a novel implementation based on realistic neuronal dynamics that removes also this obstacle. By a performance that scales favourably compared to all standard clustering algorithms, unbiased large data analysis becomes feasible on standard desktop hardware.

Parameter inference is a fundamental problem in data-driven modeling. Given observed data that is believed to be a realization of some parameterized model, the aim is to find parameter values that are able to explain the observed data. In many situations, the dominant sources of uncertainty must be included into the model for making reliable predictions. This naturally leads to stochastic models. Stochastic models render parameter inference much harder, as the aim then is to find a distribution of likely parameter values. In Bayesian statistics, which is a consistent framework for data-driven learning, this so-called posterior distribution can be used to make probabilistic predictions. We propose a novel, exact, and very efficient approach for generating posterior parameter distributions for stochastic differential equation models calibrated to measured time series. The algorithm is inspired by reinterpreting the posterior distribution as a statistical mechanics partition function of an object akin to a polymer, where the measurements are mapped on heavier beads compared to those of the simulated data. To arrive at distribution samples, we employ a Hamiltonian Monte Carlo approach combined with a multiple time-scale integration. A separation of time scales naturally arises if either the number of measurement points or the number of simulation points becomes large. Furthermore, at least for one-dimensional problems, we can decouple the harmonic modes between measurement points and solve the fastest part of their dynamics analytically. Our approach is applicable to a wide range of inference problems and is highly parallelizable.

The General Unified Threshold model of Survival (GUTS) provides a consistent mathematical framework for survival analysis. However, the calibration of GUTS models is computationally challenging. We present a novel algorithm and its fast implementation in our R package, GUTS, that help to overcome these challenges. We show a step-by-step application example consisting of model calibration and uncertainty estimation as well as making probabilistic predictions and validating the model with new data. Using self-defined wrapper functions, we show how to produce informative text printouts and plots without effort, for the inexperienced as well as the advanced user. The complete ready-to-run script is available as supplemental material. We expect that our software facilitates novel re-analysis of existing survival data as well as asking new research questions in a wide range of sciences. In particular the ability to quickly quantify stressor thresholds in conjunction with dynamic compensating processes, and their uncertainty, is an improvement that complements current survival analysis methods.

The General Unified Threshold model for Survival (GUTS) integrates previously published toxicokinetic-toxicodynamic models and estimates survival with explicitly defined assumptions. Importantly, GUTS accounts for time-variable exposure to the stressor. We performed three studies to test the ability of GUTS to predict survival of aquatic organisms across different pesticide exposure patterns, time scales and species. Firstly, using synthetic data, we identified experimental data requirements which allow for the estimation of all parameters of the GUTS proper model. Secondly, we assessed how well GUTS, calibrated with short-term survival data of Gammarus pulex exposed to four pesticides, can forecast effects of longer-term pulsed exposures. Thirdly, we tested the ability of GUTS to estimate 14-day median effect concentrations of malathion for a range of species and use these estimates to build species sensitivity distributions for different exposure patterns. We find that GUTS adequately predicts survival across exposure patterns that vary over time. When toxicity is assessed for time-variable concentrations species may differ in their responses depending on the exposure profile. This can result in different species sensitivity rankings and safe levels. The interplay of exposure pattern and species sensitivity deserves systematic investigation in order to better understand how organisms respond to stress, including humans.

Rainfall input uncertainty is one of the major concerns in hydrological modeling. Unfortunately, during inference, input errors are usually neglected, which can lead to biased parameters and implausible predictions. Rainfall multipliers can reduce this problem but still fail when the observed input (precipitation) has a different temporal pattern from the true one or if the true nonzero input is not detected. In this study, we propose an improved input error model which is able to overcome these challenges and to assess and reduce input uncertainty. We formulate the average precipitation over the watershed as a stochastic input process (SIP) and, together with a model of the hydrosystem, include it in the likelihood function. During statistical inference, we use “noisy” input (rainfall) and output (runoff) data to learn about the “true” rainfall, model parameters, and runoff. We test the methodology with the rainfall-discharge dynamics of a small urban catchment. To assess its advantages, we compare SIP with simpler methods of describing uncertainty within statistical inference: (i) standard least squares (LS), (ii) bias description (BD), and (iii) rainfall multipliers (RM). We also compare two scenarios: accurate versus inaccurate forcing data. Results show that when inferring the input with SIP and using inaccurate forcing data, the whole-catchment precipitation can still be realistically estimated and thus physical parameters can be “protected” from the corrupting impact of input errors. While correcting the output rather than the input, BD inferred similarly unbiased parameters. This is not the case with LS and RM. During validation, SIP also delivers realistic uncertainty intervals for both rainfall and runoff. Thus, the technique presented is a significant step toward better quantifying input uncertainty in hydrological inference. As a next step, SIP will have to be combined with a technique addressing model structure uncertainty.

Many simulation-intensive tasks in the applied sciences, such as sensitivity analysis, parameter inference or real time control, are hampered by slow simulators. Emulators provide the opportunity of speeding up simulations at the cost of introducing some inaccuracy. An emulator is a fast approximation to a simulator that interpolates between design input–output pairs of the simulator. Increasing the number of design data sets is a computationally demanding way of improving the accuracy of emulation. We investigate the complementary approach of increasing emulation accuracy by including knowledge about the mechanisms of the simulator into the formulation of the emulator. To approximately reproduce the output of dynamic simulators, we consider emulators that are based on a system of linear, ordinary or partial stochastic differential equations with a noise term formulated as a Gaussian process of the parameters to be emulated. This stochastic model is then conditioned to the design data so that it mimics the behavior of the nonlinear simulator as a function of the parameters. The drift terms of the linear model are designed to provide a simplified description of the simulator as a function of its key parameters so that the required corrections by the conditioned Gaussian process noise are as small as possible. The goal of this paper is to compare the gain in accuracy of these emulators by enlarging the design data set and by varying the degree of simplification of the linear model. We apply this framework to a simulator for the shallow water equations in a channel and compare emulation accuracy for emulators based on different spatial discretization levels of the channel and for a standard non-mechanistic emulator. Our results indicate that we have a large gain in accuracy already when using the simplest mechanistic description by a single linear reservoir to formulate the drift term of the linear model. Adding some more reservoirs does not lead to a significant improvement in accuracy. However, the transition to a spatially continuous linear model leads again to a similarly large gain in accuracy as the transition from the non-mechanistic emulator to that based on one reservoir.

Gaussian process (GP) emulation is a data-driven method that substitutes a slow simulator with a stochastic approximation. It is then typically orders of magnitude faster than the simulator at the costs of introducing interpolation errors. Our approach, the mechanism-based GP emulator, uses knowledge of the simulator mechanisms in addition to the information gained from previous simulator runs, so called design data. In this study, we investigate how the degree of incorporating mechanisms into the design of the GP emulator influences emulation accuracy. Similarly to the previous results, we get a significant gain in accuracy already when using the simplest approximation of the mechanisms by a single linear reservoir. However, in this case, we again considerably improve emulation accuracy when using the next two approximations. This allows us to decreases the required number of design data to achieve a similar accuracy as a non-mechanistic emulator.

We propose that a handle could be put on big data by looking at the systems that actually generate the data, rather than the data itself, realizing that there may be only few generic processes involved in this, each one imprinting its very specific structures in the space of systems, the traces of which translate into feature space. From this, we propose a practical computational clustering approach, optimized for coping with such data, inspired by how the human cortex is known to approach the problem.

Approximate Bayes computations (ABC) are used for parameter inference when the likelihood function of the model is expensive to evaluate but relatively cheap to sample from. In particle ABC, an ensemble of particles in the product space of model outputs and parameters is propagated in such a way that its output marginal approaches a delta function at the data and its parameter marginal approaches the posterior distribution. Inspired by Simulated Annealing, we present a new class of particle algorithms for ABC, based on a sequence of Metropolis kernels, associated with a decreasing sequence of tolerances w.r.t. the data. Unlike other algorithms, our class of algorithms is not based on importance sampling. Hence, it does not suffer from a loss of effective sample size due to re-sampling. We prove convergence under a condition on the speed at which the tolerance is decreased. Furthermore, we present a scheme that adapts the tolerance and the jump distribution in parameter space according to some mean-fields of the ensemble, which preserves the statistical independence of the particles, in the limit of infinite sample size. This adaptive scheme aims at converging as close as possible to the correct result with as few system updates as possible via minimizing the entropy production of the process. The performance of this new class of algorithms is compared against two other recent algorithms on two toy examples as well as on a real-world example from genetics.

Hydrological or drainage models can be valuable tools to support water management in urban environments. They can help to assess the characteristics of a catchment and to make predictions about its response (e.g. discharge). Unfortunately, parameters and results of these models are not perfect. These inaccuracies manifest themselves as model discrepancy, the so-called bias. Model bias represents the systematic deviation between model output and the real system response. There are two main causes of bias. First, there are errors in the input estimates, e.g. due to the insufficient coverage of pluviometers. Second, there are deficits in the model structure, e.g. due to the use of oversimplified empirical equations. Besides model bias, there are output measurement errors, e.g. due to imprecise flowmeters.
Until now, a large proportion of hydrological studies implicitly neglected input and structural errors. Neglecting these can lead to: i) misrepresented output measurement errors, ii) incorrect parameter estimates which compensate for model discrepancy, and iii) predictions which underestimate the uncertainty. Furthermore, these approaches do not provide guidance for finding the reasons for bias. Therefore, they cannot support its reduction. This inappropriate error consideration can finally lead to faulty risk assessment, flawed evaluation of decision alternatives, and, consequently, impaired urban water management. More advanced error assessment techniques are therefore called for.
The main goal of this thesis is to better represent input, structural, and output measurement errors. This can meliorate parameter estimation and prediction generation. The main thesis contributions are two. The first is a realistic description of the uncertainties in stormwater, wastewater, and sediment transport predictions. The second is a sound representation of errors in the rainfall inputs. Additionally, we discuss and compare different statistical methods used for hydrological inference. Finally, we also present a new tool to improve sewer flow monitoring. [...]

Oversimplified models and erroneous inputs play a significant role in impairing environmental predictions. To assess the contribution of these errors to model uncertainties is still challenging. Our objective is to understand the effect of model complexity on systematic modeling errors. Our method consists of formulating alternative models with increasing detail and flexibility and describing their systematic deviations by an autoregressive bias process. We test the approach in an urban catchment with five drainage models. Our results show that a single bias description produces reliable predictions for all models. The bias decreases with increasing model complexity and then stabilizes. The bias decline can be associated with reduced structural deficits, while the remaining bias is probably dominated by input errors. Combining a bias description with a multimodel comparison is an effective way to assess the influence of structural and rainfall errors on flow forecasts.

Flow cytometry (FC) devices count and measure cells in fluids in an automated procedure. In this paper we present our work in progress on the clustering of FC data. We compare standard clustering algorithms such as K-means, Ward’s clustering, etc., to the more advanced approach of sequential superparamagnetic clustering (SSC). We found Ward’s hierarchical clustering to perform best regarding internal cluster validation measures, while SSC yielded the best results based on the visual inspection of the clustering results.

Environmental modeling often requires combining prior knowledge with information obtained from data. The robust Bayesian approach makes it possible to consider ambiguity in this prior knowledge. Describing such ambiguity using sets of probability distributions defined by the Density Ratio Class has important conceptual advantages over alternative robust formulations. Earlier studies showed that the Density Ratio Class is invariant under Bayesian inference and marginalization. We prove that (i) the Density Ratio Class is also invariant under propagation through deterministic models, whereas (ii) predictions of a stochastic model with parameters defined by a Density Ratio Class are embedded in a Density Ratio Class. These invariance properties make it possible to describe sequential learning and prediction under a unified framework. We developed numerical algorithms to minimize the additional computational burden relative to the use of single priors. Practical feasibility of these methods is demonstrated by their application to a simple ecological model.

Hydrodynamische Kanalnetzmodelle sind für die Entwässerungsplanung unersetzlich.Obwohl sie den komplexen Niederschlags-Abfluss Prozess niemals exakt abbilden können, liefern sie oft akzeptable Ergebnisse. Es ist jedoch schwierig, plausible Schätzungen der Unsicherheit von Kanalnetzberechnungen zu erhalten, da die gängigen statistischen Verfahren auf sehr starken Annahmen über die Fehlerverteilung beruhen, wie z.B. unabhängige, und identisch verteilte Zufallseffekte, welche praktisch nie erfüllt sind. In diesem Artikel stellen wir deshalb eine Methode vor, um i) zuverlässigere Schätzungen der Unsicherheit von Kanalnetzsimulationen zu berechnen und ii) die Vorhersageunsicherheit in einzelne Komponenten zu trennen. Zu diesem Zweck werden neben dem deterministischen Modell und zufälligen Messfehlern auch systematische Modellabweichungen berücksichtigt. Diese systematischen Modellabweichungen werden durch einen autoregressiven Zufallsprozess beschrieben und erfassen die Effekte von Unsicherheiten in den Eingangsdaten und der Modellstruktur. Zur Kalibration wird Bayessche Parameterschätzung verwendet. Wir zeigen an einem Fallbeispiel, dass die Berücksichtigung der Modellabweichung die Zuverlässigkeit der Berechnungsergebnisse stark verbessert. Ausserdem wird deutlich, dass systematische Modellabweichungen die Vorhersageunsicherheit dominieren und weit grösser sind, als Unsicherheit in den Modellparametern oder Messfehler. Abgesehen von den langen Rechenzeiten ergeben sich gewisse Limitierungen aus der subjektiven Formulierung des Vorwissens über die Parameter des Fehlermodells. Die statistische Berücksichtigung von systematischen Modellabweichungen liefert jedoch eine zuverlässige Schätzungen der Unsicherheit als herkömmliche Methoden und scheint daher für Kanalnetzsimulationen sehr interessant.

Hydrodynamic models are useful tools for urban water management. Unfortunately, it is still challenging to obtain accurate results and plausible uncertainty estimates when using these models. In particular, with the currently applied statistical techniques, flow predictions are usually overconfident and biased. In this study, we present a flexible and relatively efficient methodology (i) to obtain more reliable hydrological simulations in terms of coverage of validation data by the uncertainty bands and (ii) to separate prediction uncertainty into its components. Our approach acknowledges that urban drainage predictions are biased. This is mostly due to input errors and structural deficits of the model. We address this issue by describing model bias in a Bayesian framework. The bias becomes an autoregressive term additional to white measurement noise, the only error type accounted for in traditional uncertainty analysis. To allow for bigger discrepancies during wet weather, we make the variance of bias dependent on the input (rainfall) or/and output (runoff) of the system. Specifically, we present a structured approach to select, among five variants, the optimal bias description for a given urban or natural case study. We tested the methodology in a small monitored stormwater system described with a parsimonious model. Our results clearly show that flow simulations are much more reliable when bias is accounted for than when it is neglected. Furthermore, our probabilistic predictions can discriminate between three uncertainty contributions: parametric uncertainty, bias, and measurement errors. In our case study, the best performing bias description is the output-dependent bias using a log-sinh transformation of data and model results. The limitations of the framework presented are some ambiguity due to the subjective choice of priors for bias parameters and its inability to address the causes of model discrepancies. Further research should focus on quantifying and reducing the causes of bias by improving the model structure and propagating input uncertainty.

In applied sciences, we often deal with deterministic simulation models that are too slow for simulationintensive tasks such as calibration or real-time control. In this paper, an emulator for a generic dynamic model, given by a system of ordinary non-linear differential equations, is developed. The non-linear differential equations are linearized and Gaussian white noise is added to account for the non-linearities. The resulting linear stochastic system is conditioned on a set of solutions of the non-linear equations that have been calculated prior to the emulation. A path-integral approach is used to derive the Gaussian distribution of the emulated solution. The solution reveals that most of the computational burden can be shifted to the conditioning phase of the emulator and the complexity of the actual emulation step only scales like O(Nnm²), where N is the number of time-points at which the solution is to be emulated, n the number of solutions the emulator is conditioned on and m the number of variables.), where N is the number of time-points at which the solution is to be emulated, n the number of solutions the emulator is conditioned on and m the number of variables.

In applied sciences, we often deal with deterministic simulation models that are too slow for simulation-intensive tasks such as calibration or real-time control. In this paper, an emulator for a generic dynamic model, given by a system of ordinary nonlinear differential equations, is developed. The nonlinear differential equations are linearized and Gaussian white noise is added to account for the nonlinearities. The resulting linear stochastic system is conditioned on a set of solutions of the nonlinear equations that have been calculated prior to the emulation. A path-integral approach is used to derive the Gaussian distribution of the emulated solution. The solution reveals that most of the computational burden can be shifted to the conditioning phase of the emulator and the complexity of the actual emulation step only scales like O(Nn) in multiplications of matrices of the dimension of the state space. Here, N is the number of time-points at which the solution is to be emulated and n is the number of solutions the emulator is conditioned on.
The applicability of the algorithm is demonstrated with the hydrological model logSPM.

The aim of this study is to apply the Bayesian method of identifying optimal experimental designs to a toxicokinetic–toxicodynamic model that describes the response of aquatic organisms to time dependent concentrations of toxicants. As for experimental designs, we restrict ourselves to pulses and constant concentrations. A design of an experiment is called optimal within this set of designs if it maximizes the expected gain of knowledge about the parameters. Focus is on parameters that are associated with the auxiliary damage variable of the model that can only be inferred indirectly from survival time series data. Gain of knowledge through an experiment is quantified both with the ratio of posterior to prior variances of individual parameters and with the entropy of the posterior distribution relative to the prior on the whole parameter space. The numerical methods developed to calculate expected gain of knowledge are expected to be useful beyond this case study, in particular for multinomially distributed data such as survival time series data.

We report on the advantages and problems of using toxicokinetic-toxicodynamic (TKTD) models for the analysis, understanding, and simulation of sublethal effects. Only a few toxicodynamic approaches for sublethal effects are available. These differ in their effect mechanism and emphasis on linkages between endpoints. We discuss how the distinction between quantal and graded endpoints and the type of linkage between endpoints can guide model design and selection. Strengths and limitations of two main approaches and possible ways forward are outlined.

Toxicokinetic-toxicodynamic models (TKTD models) simulate the time-course of processes leading to toxic effects on organisms. Even for an apparently simple endpoint as survival, a large number of very different TKTD approaches exist. These differ in their underlying hypotheses and assumptions, although often the assumptions are not explicitly stated. Thus, our first objective was to illuminate the underlying assumptions (individual tolerance or stochastic death, speed of toxicodynamic damage recovery, threshold distribution) of various existing modeling approaches for survival and show how they relate to each other (e.g., critical body residue, critical target occupation, damage assessment, DEBtox survival, threshold damage). Our second objective was to develop a general unified threshold model for survival (GUTS), from which a large range of existing models can be derived as special cases. Specific assumptions to arrive at these special cases are made and explained. Finally, we illustrate how special cases of GUTS can be fitted to survival data. We envision that GUTS will help increase the application of TKTD models in ecotoxicological research as well as environmental risk assessment of chemicals. It unifies a wide range of previously unrelated approaches, clarifies their underlying assumptions, and facilitates further improvement in the modeling of survival under chemical stress.

The Batalin-Vilkovisky (BV) method is the most powerful method to analyze functional integrals with (infinite-dimensional) gauge symmetries presently known. It has been invented to fix gauges associated with symmetries that do not close off-shell. Homological perturbation theory is introduced and used to develop the integration theory behind BV and to describe the BV quantization of a Lagrangian system with symmetries. Localization (illustrated in terms of Duistermaat-Heckman localization) as well as anomalous symmetries are discussed in the framework of BV.

Currently a cosmic ray cloud connection (CRC) hypothesis is subject of an intense controversial debate. It postulates that galactic cosmic rays (GCR) intruding the Earth's atmosphere influence cloud cover. If correct it would have important consequences for our understanding of climate driving processes. Here we report on an alternative and stringent test of the CRC-hypothesis by searching for a possible influence of sudden GCR decreases (so-called Forbush decreases) on clouds. We find no response of global cloud cover to Forbush decreases at any altitude and latitude.