Workshop
A causal view on dynamical systems
Sören Becker · Alexis Bellot · Cecilia Casolo · Niki Kilbertus · Sara Magliacane · Yuyang (Bernie) Wang
Room 387
Schedule
Sat 7:00 a.m.  7:05 a.m.

Opening words
(
Introduction
)
SlidesLive Video 
🔗 
Sat 7:05 a.m.  7:30 a.m.

Towards Markov Properties for ContinuousTime Dynamical Systems  Joris Mooij
(
Keynote Talk
)
SlidesLive Video 
Joris Mooij 🔗 
Sat 7:30 a.m.  7:34 a.m.

Q&A
(
Q&A
)

🔗 
Sat 7:34 a.m.  7:35 a.m.

Speaker Introduction
(
Speaker Introduction
)

🔗 
Sat 7:35 a.m.  8:00 a.m.

Causal and graphical models for continuoustime event data
(
Keynote Talk
)
SlidesLive Video 
Vanessa Didelez 🔗 
Sat 8:00 a.m.  8:05 a.m.

Q&A Causal and graphical models for continuoustime event data
(
Q&A
)

🔗 
Sat 8:05 a.m.  8:29 a.m.

Coffee Break
(
Break
)

🔗 
Sat 8:29 a.m.  8:30 a.m.

Speaker Introduction
(
Speaker Introduction
)

🔗 
Sat 8:30 a.m.  8:55 a.m.

Dynamic Causal Modelling
(
Keynote Speaker Talk
)
SlidesLive Video 
Karl Friston 🔗 
Sat 8:55 a.m.  9:00 a.m.

Q&A Dynamic Causal Modelling
(
Q&A
)

🔗 
Sat 9:00 a.m.  9:25 a.m.

Causal feature selection in time series data
(
Keynote Talk
)
SlidesLive Video 
Atalanti Mastakouri 🔗 
Sat 9:25 a.m.  9:30 a.m.

Q&A Causal feature selection in time series data
(
Q&A
)

🔗 
Sat 9:30 a.m.  10:30 a.m.

Poster Session
(
Poster Session
)

🔗 
Sat 10:30 a.m.  11:59 a.m.

Lunch Break
(
Break
)

🔗 
Sat 11:59 a.m.  12:00 p.m.

Speaker Introduction
(
Speaker Introduction
)

🔗 
Sat 12:00 p.m.  12:25 p.m.

Signature kernel methods
(
Keynote Talk
)
SlidesLive Video 
Cristopher Salvi 🔗 
Sat 12:25 p.m.  12:30 p.m.

Q&A Signature kernel methods
(
Q&A
)

🔗 
Sat 12:30 p.m.  12:55 p.m.

Causal Discovery from Nonstationary Time Series  Biwei Huang
(
Keynote Talk
)
SlidesLive Video 
Biwei Huang 🔗 
Sat 12:55 p.m.  1:00 p.m.

Q&A
(
Q&A
)

🔗 
Sat 1:00 p.m.  1:30 p.m.

Coffee Break
(
Break
)

🔗 
Sat 1:30 p.m.  1:42 p.m.

Causal Inference out of Control: Identifying the Steerability of Consumption
(
Oral
)
link
SlidesLive Video Regulators and academics are increasingly interested in the causal effect that algorithmic actions of a digital platform have on consumption. We introduce a general causal inference problem we call the steerability of consumption that abstracts many settings of interest. Focusing on observational designs, we exhibit a set of assumptions for identifiability that significantly weakens the often unrealistic coverage assumptions of standard designs. They key insight behind our assumptions is to model the dynamics of consumption, viewing the platform as a controller acting on a dynamical system. From this dynamical systems perspective, we are able to show that exogenous variation in consumption and appropriately responsive control actions are sufficient for indentifying steerability of consumption. Our results illustrate the fruitful interplay of control theory and causal inference, which we illustrate with examples from econometrics, macroeconomics, and machine learning. 
Gary Cheng · Moritz Hardt · Celestine MendlerDünner 🔗 
Sat 1:42 p.m.  1:54 p.m.

A Balanced Design of Time Series Experiments
(
Oral
)
link
SlidesLive Video Time series experiments are a family of experimental designs on a time series. One experimental unit is sequentially exposed to some version of treatment, stays in the version of treatment for a duration of time, and gets exposed to another version of treatment. While this type of experimental design could handle population interference between units, it typically still needs to account for temporal interference, i.e., a treatment at an earlier period persists in impacting the outcomes of the later periods. Practitioners have widely recognized the applicability of time series experiments, yet prior work typically requires a long duration to gain enough power. In this paper, we propose a novel randomized design that significantly increases the power of such experiments. We prove the theoretical performance of the novel design and verify its superior performance by conducting an extensive simulation study. 
Tu Ni · Iavor Bojinov · Jinglong Zhao 🔗 
Sat 1:54 p.m.  2:06 p.m.

GRACEC: Generalized Rate Agnostic Causal Estimation via Constraints
(
Oral
)
link
SlidesLive Video Graphical structures estimated by causal learning algorithms from time series data can provide highly misleading causal information if the causal timescale of the generating process fails to match the measurement timescale of the data. Existing algorithms provide limited resources to respond to this challenge, and so researchers must either use models that they know are likely misleading, or else forego causal learning entirely. Existing methods face uptofour distinct shortfalls, as they might a) require that the difference between causal and measurement timescales is known; b) only handle very small number of random variables when the timescale difference is unknown; c) only apply to pairs of variables (albeit with fewer assumptions about prior knowledge); or d) be unable to find a solution given statistical noise in the data. This paper addresses all four challenges. Our algorithm combines constraint programming with both theoretical insights into the problem structure and prior information about admissible causal interactions to achieve gains of multiple orders of magnitude in speed and informativeness. The resulting system scales to significantly larger sets of random variables (>100) without knowledge of the timescale difference while maintaining theoretical guarantees. This method is also robust to edge misidentification and can use parametric connection strengths, while optionally finding the optimal among many possible solutions. 
Mohammadsajad Abavisani · David Danks · Vince Calhoun · Sergey Plis 🔗 
Sat 2:06 p.m.  2:20 p.m.

Learning Heterogeneous Interaction Strengths by Trajectory Prediction with Graph Neural Network
(
Poster
)
link
SlidesLive Video Dynamical systems with interacting agents are universal in nature, commonly modeled by a graph of relationships between their constituents. Recently, various works have been presented to tackle the problem of inferring those relationships from the system trajectories via deep neural networks, but most of the studies assume binary or discrete types of interactions for simplicity. In the real world, the interaction kernels often involve continuous interaction strengths, which cannot be accurately approximated by discrete relations. In this work, we propose the relational attentive inference network (RAIN) to infer continuously weighted interaction graphs without any groundtruth interaction strengths. Our model employs a novel pairwise attention (PA) mechanism to refine the trajectory representations and a graph transformer to extract heterogeneous interaction weights for each pair of agents. We show that our RAIN model with the PA mechanism accurately infers continuous interaction strengths for simulated physical systems in an unsupervised manner. Further, RAIN with PA successfully predicts trajectories from motion capture data with an interpretable interaction graph, demonstrating the virtue of modeling unknown dynamics with continuous weights. 
Seungwoong Ha · Hawoong Jeong 🔗 
Sat 2:20 p.m.  2:50 p.m.

Poster Session
(
Poster Session
)

🔗 
Sat 2:50 p.m.  3:00 p.m.

Closing Words
(
Closing Words
)

🔗 


Latent GPODEs with Informative Priors
(
Poster
)
link
For many complex systems the parametric form of the differential equation might be unknown or infeasible to determine. Earlier works have explored to model the unknown ODE system with a Gaussian Process model, however, the application has been limited to a low dimensional data setting. We propose a novel framework by combining a generative and a Bayesian nonparametric model. Our model learns a physically meaningful latent representation (position, momentum) and solves in the latent space an ODE system. The use of GP allows us to account for uncertainty as well as to extend our work with informative priors. We demonstrate our framework on an image rotation dataset. The method demonstrates its ability to learn dynamics from high dimensional data and we obtain stateoftheart performance compared to earlier GPbased ODEs models on dynamic forecasting. 
Ilze Amanda Auzina · Çağatay Yıldız · Efstratios Gavves 🔗 


Estimating the mechanisms underlying transient dynamics based on perievent data
(
Poster
)
link
Many important dynamical phenomena emerging in complex systems such as storms, stock market crashes, or reactivations of memory engrams in the mammalian brain are transient in nature. We consider the problem of learning accurate models of such phenomena based only on data gathered by detecting such transient events, and analyzing their perievent dynamics. This approach is widely used to analyze spontaneous activity in brain recording, as it focuses on emerging events of particular significance to brain function. We show, however, that such an approach may misrepresent the properties of the system under study due to the event detection procedure that entails a selection bias. We develop the Debiased Snapshot (DeSnap) approach to debias the timevarying properties of the system estimated from such perievent data and demonstrate its benefits in recovering statedependent transient dynamics in toy examples and neural time series. 
Kaidi Shao · Nikos K Logothetis · Michel Besserve 🔗 


Reducing Causal Illusions through Deliberate Undersampling
(
Poster
)
link
Domain scientists interested in the causal mechanisms are usually limited by the frequency at which they can collect the measurements of social, physical, or biological systems.It is a reasonable assumption that higher frequency is more informative of the causal structure.This assumption is a strong driver for designing new, faster instruments.A task that is expensive and often impossible at the current state of technology.In this work, we show that counter to the intuition it is possible for causal systems to improve the estimation of causal graphs from undersampled timeseries by augmenting the measurements with those collected at a rate slower than currently available.We present an algorithm able to take advantage of measurement timescale graphs estimated from data at various sampling rates and lower the underdeterminacy of the system by reducing the equivalence size.We investigate the probability of cases in which deliberate undersampling yields a gain and the size of this gain. 
Kseniya Solovyeva · David Danks · Mohammadsajad Abavisani · Sergey Plis 🔗 


Causal Inference out of Control: Identifying the Steerability of Consumption
(
Poster
)
link
Regulators and academics are increasingly interested in the causal effect that algorithmic actions of a digital platform have on consumption. We introduce a general causal inference problem we call the steerability of consumption that abstracts many settings of interest. Focusing on observational designs, we exhibit a set of assumptions for identifiability that significantly weakens the often unrealistic coverage assumptions of standard designs. They key insight behind our assumptions is to model the dynamics of consumption, viewing the platform as a controller acting on a dynamical system. From this dynamical systems perspective, we are able to show that exogenous variation in consumption and appropriately responsive control actions are sufficient for indentifying steerability of consumption. Our results illustrate the fruitful interplay of control theory and causal inference, which we illustrate with examples from econometrics, macroeconomics, and machine learning. 
Gary Cheng · Moritz Hardt · Celestine MendlerDünner 🔗 


Bayesian Dynamic Causal Discovery
(
Poster
)
link
Learning the causal structure of observable variables is a central focus for scientific discovery. Bayesian causal discovery methods tackle this problem by learning a posterior over the set of admissible graphs that are equally likely given our priors and observations. Existing methods primarily consider observations from static systems and assume the underlying causal structure takes the form of a directed acyclic graph (DAG). In settings with dynamic feedback mechanisms that regulate the trajectories of individual variables, this acyclicity assumption fails unless we account for time. We treat causal discovery in the unrolled causal graph as a problem of sparse identification of a dynamical system. This imposes a natural temporal causal order between variables and captures cyclic feedback loops through time. Under this lens, we propose a new framework for Bayesian causal discovery for dynamical systems and present a novel generative flow network architecture (DynGFN) tailored for this task. DynGFN imposes an edgewise sparse prior to sequentially build a $k$sparse causal graph. Through evaluation on temporal data, our results show that the posterior learned with DynGFN yields improved Bayes coverage of admissible causal structures relative to state of the art Bayesian causal discovery methods.

Alexander Tong · Lazar Atanackovic · Jason Hartford · Yoshua Bengio 🔗 


Spatiotemporal Information Flows
(
Poster
)
link
We develop a novel flow extraction framework, \textit{spatiotemporal information flow} to capture, in an unbiased manner, salient causal relationships between pixels over space and time. Real spatiotemporal dynamical systems such as cellular morphodynamics are complex, nonlinear and evolve over time in response to feedbacks. This makes it highly challenging to model, simulate or fit phenomena from first principle physics. More critically, we often do not know \textit{a priori} and desire to discover the salient variables to include and the key relationships to model. As such causal measures to identify relationships direct from observational multivariate timeseries have been developed. These measures however have largely only been studied in 1D. Here, spatiotemporal information flows present a general, multiscale method to extend 1D causal measures to 2D + time video. Applying spatiotemporal information flows we discover the salient pixeltopixel information transfer highways in videos of diverse phenomena from traffic and crowd flow, to collision physics, to fish swarming, to moving camouflaged animals, to human action, embryo development, cell division and cell migration. 
Felix Zhou · Roshan Ravishankar 🔗 


GRACEC: Generalized Rate Agnostic Causal Estimation via Constraints
(
Poster
)
link
SlidesLive Video Graphical structures estimated by causal learning algorithms from time series data can provide highly misleading causal information if the causal timescale of the generating process fails to match the measurement timescale of the data. Existing algorithms provide limited resources to respond to this challenge, and so researchers must either use models that they know are likely misleading, or else forego causal learning entirely. Existing methods face uptofour distinct shortfalls, as they might a) require that the difference between causal and measurement timescales is known; b) only handle very small number of random variables when the timescale difference is unknown; c) only apply to pairs of variables (albeit with fewer assumptions about prior knowledge); or d) be unable to find a solution given statistical noise in the data. This paper addresses all four challenges. Our algorithm combines constraint programming with both theoretical insights into the problem structure and prior information about admissible causal interactions to achieve gains of multiple orders of magnitude in speed and informativeness. The resulting system scales to significantly larger sets of random variables (>100) without knowledge of the timescale difference while maintaining theoretical guarantees. This method is also robust to edge misidentification and can use parametric connection strengths, while optionally finding the optimal among many possible solutions. 
Mohammadsajad Abavisani · David Danks · Vince Calhoun · Sergey Plis 🔗 


Causal Bandits: Online DecisionMaking in Endogenous Settings
(
Poster
)
link
The deployment of MultiArmed Bandits (MAB) has become commonplace in many economic applications. However, regret guarantees for even stateoftheart linear bandit algorithms (such as Optimism in the Face of Uncertainty Linear bandit (OFUL)) make strong exogeneity assumptions w.r.t. arm covariates. This assumption is very often violated in many economic contexts and using such algorithms can lead to suboptimal decisions. In this paper, we consider the problem of online learning in linear stochastic multiarmed bandit problems with endogenous covariates. We propose an algorithm we term BanditIV, that uses instrumental variables to correct for this bias, and prove an $\tilde{\mathcal{O}}(k\sqrt{T})$ upper bound for the expected regret of the algorithm. Further, in economic contexts, it is also important to understand how the model parameters behave asymptotically. To this end, we additionally propose $\epsilon$BanditIV algorithm and demonstrate its asymptotic consistency and normality while ensuring the same regret bound. Finally, we carry out extensive Monte Carlo simulations to demonstrate the performance of our algorithms compared to other methods. We show that BanditIV and $\epsilon$BanditIV significantly outperform other existing methods.

Jingwen Zhang · Yifang Chen · Amandeep Singh 🔗 


Learning Mixture Structure on MultiSource Time Series for Probabilistic Forecasting
(
Poster
)
link
In datadriven applications, it is increasingly desirable to collect data from different sources for enhancing performance.In this paper, we are interested in the problem of probabilistic forecasting with multisource time series.We propose a neural mixture structurebased probability model for learning different predictive relations and their adaptive combinations from multisource time series.We present the prediction and uncertainty quantification methods, which are applicable to different distributions of target variables.Additionally, given the imbalanced and unstable behaviors observed during the direct training of the proposed mixture model, we develop a phased learning method and provide a theoretical analysis.In the experimental evaluation, the mixture model trained by the phased learning exhibits competitive performance on both point and probabilistic prediction metrics.Meanwhile, the proposed uncertainty conditioned error suggests the potential of the mixture model's uncertainty score as a reliability indicator of predictions. 
Tian Guo 🔗 


Towards Reverse Causal Inference on Panel Data: Precise Formulation and Challenges
(
Poster
)
link
Seeking causal explanations in panel (or longitudinal/multivariate timeseries) data is a difficult problem of both academic and industrial importance. Although there exists huge literature on forward causal inference where the treatment/outcome/covariates are welldefined, it is unclear how to answer the reverse question: which covariates have effects on the outcome? In this paper, we set forth our expedition on this reverse question from the first principles. We formulate the precise problem definition in terms of causal patterns and causal paths, propose a lineartime greedy algorithm that makes use of forward causal inference estimators, and identify a set of optimality conditions under which the proposed algorithm is able to find the best causal path. To substantiate our meta algorithm, we propose a generalized version of the synthetic control estimator by fitting both synthetic treatments and controls by conditioning on the partial causal paths. We perform simulation studies on synthetic datasets and demonstrate the potential of our method. 
Jiayao Zhang · Youngsuk Park · Danielle Maddix · Dan Roth · Yuyang (Bernie) Wang 🔗 


Approximate Uncertainty Propagation for Continuous Gaussian Process Dynamical Systems
(
Poster
)
link
One of the benefits of using Gaussian Process models is the availability of uncertainty quantification. However, when learning continuous dynamical systems obtaining trajectories requires repeatedly mapping uncertain inputs through the learned nonlinear function, which is generally nontractable. As samplingbased approaches are computationally expensive, we consider approximations of the output and trajectory distribution. We show that existing approaches make an incorrect implicit independence assumption and underestimate the modelinduced uncertainty. We propose a piecewise linear approximation of the GP model and a numerical solver for efficient uncertainty estimates matching samplingbased methods at a lower computational cost. 
Steffen Ridderbusch · Sina OberBlöbaum · Paul Goulart 🔗 


A Balanced Design of Time Series Experiments
(
Poster
)
link
Time series experiments are a family of experimental designs on a time series. One experimental unit is sequentially exposed to some version of treatment, stays in the version of treatment for a duration of time, and gets exposed to another version of treatment. While this type of experimental design could handle population interference between units, it typically still needs to account for temporal interference, i.e., a treatment at an earlier period persists in impacting the outcomes of the later periods. Practitioners have widely recognized the applicability of time series experiments, yet prior work typically requires a long duration to gain enough power. In this paper, we propose a novel randomized design that significantly increases the power of such experiments. We prove the theoretical performance of the novel design and verify its superior performance by conducting an extensive simulation study. 
Tu Ni · Iavor Bojinov · Jinglong Zhao 🔗 


Time Series Causal Link Estimation under Hidden Confounding using Knockoff Interventions
(
Poster
)
link
Latent variables often mask causeeffect relationships in observational data which provokes spurious links that may be misinterpreted as causal. This problem sparks great interest in the fields such as climate science and economics. We propose to estimate confounded causal links of time series using Sequential Causal Effect Variational Autoencoder (SCEVAE) while applying knockoff interventions. We show the advantage of knockoff interventions by applying SCEVAE to synthetic datasets with both linear and nonlinear causal links. Moreover, we apply SCEVAE with knockoffs to real aerosolcloudclimate observational time series data. We compare our results on synthetic data to those of a time series deconfounding method both with and without estimated confounders. We show that our method outperforms this benchmark by comparing both methods to the ground truth. For the real data analysis, we rely on expert knowledge of causal links and demonstrate how using suitable proxy variables improves the causal link estimation in the presence of hidden confounders. 
VioletaTeodora Trifunov · Maha Shadaydeh · Joachim Denzler 🔗 


On the Complexity of Counterfactual Reasoning
(
Poster
)
link
A common form of counterfactual reasoning is based on the notion of twin network which is a causal graph that represents two worlds, one real and another imaginary. Information about the real world is used to update the joint distribution over the network's causal mechanisms which is then used for hypothetical reasoning in the imaginary world. This is in contrast to associational and interventional reasoning which involve a causal graph over a single world that we shall call a base network. In this paper, we study the complexity of counterfactual reasoning in twin networks in relation to the complexity of associational and interventional reasoning in base networks. We show that counterfactual reasoning is no harder than associational/interventional reasoning in the context of two computational frameworks. One of these is based on the notion of treewidth and includes the classical variable elimination and jointree algorithms. The second, more recent framework is based on the notion of causal treewidth which is directed towards causal graphs used in counterfactual reasoning (e.g., structural causal models). More specifically, we show that the (causal) treewidth of a twin network is at most twice the (causal) treewidth of its base network plus one. This means that if associational/interventional reasoning is tractable then counterfactual reasoning is also tractable. We further extend our results to a form of counterfactual reasoning that is more general than what is commonly discussed in the literature. We finally present empirical results that measure the gap between the complexities of counterfactual reasoning and associational/interventional reasoning on random networks. 
Yunqiu Han · Yizuo Chen · Adnan Darwiche 🔗 


Causal Discovery in Time Series Data Using Causally Invariant Locally Linear Models
(
Poster
)
link
Identifying causal relationship is an often desired, but difficult, task, and generally only possible under specific assumptions. In this paper we are considering the task of identifying causal relationships between entities that have a temporal axis, as for example continuous measurements of different components within a complex machine. We introduce a locally linear model class that allows us to recover causal relationships, assuming that the process is locally linear, that we have access to observations in diverse environments and that the causal structure is invariant across the different environments. We validate the model in a theoretical and two experimental settings. 
Alexander Mey 🔗 


Petri Nets Enable Causal Reasoning in Dynamical Systems
(
Poster
)
link
Dynamical systems, e.g. economic systems or biomolecular signaling networks, are processes comprised of states that evolve in time. Causal models represent these processes, and support causal queries inferring outcomes of system perturbations. Unfortunately, Structural Causal Models, the traditional causal models of choice, require the system to be in steady state and don't extend to dynamical systems. Recent formulations of causal models with a compatible dynamic syntax, such as Probability Trees, lack a semantics for representing both states and transitions of a system, limiting their ability to fully represent the system and ability to encode the underlying causal assumptions. In contrast, Petri Nets are wellstudied models of dynamical systems, with the ability to encode states and transitions. However, their use for causal reasoning has so far been underexplored. This manuscript expands the scope of causal reasoning in dynamical systems by proposing a causal semantics for Petri Nets. We define a pipeline of constructing a Petri Net model and calculating the fundamental causal queries: conditioning, interventions and counterfactuals. A novel aspect of the proposed causal semantics is an unwrapping procedure, which allows for a dichotomy of Petri Net models when calculating a query. On one hand, a base Petri Net model visually represents the system, implicitly encodes the traces defined by the system, and models the underlying causal assumptions. On the other hand, an unwrapped Petri Net explicitly represents traces, and answers causal queries of interest. We demonstrate the utility of the proposed approach on a case study of a dynamical system where Structural Causal Models fail. 
Ritwik Anand · Jeremy Zucker · Vartika Tewari · Karen Sachs · Olga Vitek 🔗 


Learning Dynamics and Structure of Complex Systems Using Graph Neural Networks
(
Poster
)
link
Many complex systems are composed of interacting parts, and the underlying laws are usually simple and universal. While graph neural networks provide a useful relational inductive bias for modeling such systems, generalization to new system instances of the same type is less studied. In this work we trained graph neural networks to fit time series from an example nonlinear dynamical system, the belief propagation algorithm. We found simple interpretations of the learned representation and model components, and they are consistent with core properties of the probabilistic inference algorithm. We successfully identified a 'graph translator' between the statistical attributes in belief propagation and parameters of the corresponding trained network, and showed that it enables two types of novel generalization: to recover the underlying structure of a new system instance based solely on time series observations, and to construct a new network from this structure directly. Our results demonstrated a path towards understanding both dynamics and structure of a complex system and how such understanding can be used for generalization. 
Zhe Li · Andreas Tolias · Xaq Pitkow 🔗 


Quantifying Causal Contribution in Rare Event Data
(
Poster
)
link
We introduce a framework for causal discovery and attribution of causal influence for rare events in time series datawhere the interest is in identifying causal links and root causes of individual discrete events rather than the types of these events. Specifically, we build on the theory of temporal point processes, and describe a discretetime analogue of Hawkes processes to model the occurrence of selfexciting rare events with instantaneous effects. We then introduce several scores to measure causal influence among individual events. These statistics are drawn from causal inference and temporal point process theories, describe complementary aspects of causality in temporal event data, and obey commonly used axioms for feature attribution. We demonstrate the efficacy of our model and the proposed influence scores on real and synthetic data. 
Caner Turkmen · Dominik Janzing · Oleksandr Shchur · Lenon Minorics · Laurent Callot 🔗 


Online Learning of Optimal Control Signals in Stochastic Linear Dynamical Systems
(
Poster
)
link
Among the most canonical systems are linear timeinvariant dynamics governed by differential equations and stochastic disturbances. An interesting problem in this class of systems is learning to minimize a quadratic cost function when system matrices are unknown. This work initiates theoretical analysis of implementable reinforcement learning policies for balancing exploration versus exploitation in such systems. We present an online policy that learns the optimal control actions fast by carefully randomizing the parameter estimates to explore. More precisely, we establish performance guarantees for the presented policy showing that the regret grows as the \emph{squareroot of time} multiplied by the \emph{number of parameters}. Implementation of the policy for a flight control task shows its efficacy. Further, we prove tight results that ensure stability under inexact system matrices and fully specify unavoidable performance degradations caused by a nonoptimal policy. To obtain the results, we conduct a novel analysis for matrix perturbation, bound comparative ratios of stochastic integrals, and introduce the new method of policy differentiation. These technical novelties are trusted to provide a useful cornerstone for continuoustime reinforcement learning. 
Mohamad Kazem Shirani Faradonbeh 🔗 


Evaluating vaccine allocation strategies using simulationassisted causal modelling
(
Poster
)
link
Early on during a pandemic, vaccine availability is limited, requiring prioritisation of different population groups. Evaluating vaccination allocation is therefore a crucial element of pandemics response. In the present work, we develop a model to retrospectively evaluate agedependent counterfactual vaccine allocation strategies against the COVID19 pandemic. To estimate the effect of allocation on the expected severecase incidence, we employ a simulationassisted causal modelling approach which combines a compartmental infectiondynamics simulation, a coarsegrained, datadriven causal model and literature estimates for immunity waning. We compare Israel's implemented vaccine allocation strategy in 2021 to counterfactual strategies such as no prioritisation, prioritisation of younger age groups or a strict riskranked approach; we find that Israel's implemented strategy was indeed highly effective. We also study the marginal impact of increasing vaccine uptake for a given age group and find that increasing vaccinations in the elderly is most effective at preventing severe cases, whereas additional vaccinations for middleaged groups reduce infections most effectively. Due to its modular structure, our model can easily be adapted to study future pandemics. We demonstrate this flexibility by investigating vaccine allocation strategies for a pandemic with characteristics of the Spanish Flu. Our approach thus helps evaluate vaccination strategies under the complex interplay of core epidemic factors, including agedependent risk profiles, immunity waning, vaccine availability and spreading rates. 
Armin Kekić · Jonas Dehning · Luigi Gresele · Julius von Kügelgen · Viola Priesemann · Bernhard Schölkopf 🔗 


Learning Heterogeneous Interaction Strengths by Trajectory Prediction with Graph Neural Network
(
Oral
)
link
Dynamical systems with interacting agents are universal in nature, commonly modeled by a graph of relationships between their constituents. Recently, various works have been presented to tackle the problem of inferring those relationships from the system trajectories via deep neural networks, but most of the studies assume binary or discrete types of interactions for simplicity. In the real world, the interaction kernels often involve continuous interaction strengths, which cannot be accurately approximated by discrete relations. In this work, we propose the relational attentive inference network (RAIN) to infer continuously weighted interaction graphs without any groundtruth interaction strengths. Our model employs a novel pairwise attention (PA) mechanism to refine the trajectory representations and a graph transformer to extract heterogeneous interaction weights for each pair of agents. We show that our RAIN model with the PA mechanism accurately infers continuous interaction strengths for simulated physical systems in an unsupervised manner. Further, RAIN with PA successfully predicts trajectories from motion capture data with an interpretable interaction graph, demonstrating the virtue of modeling unknown dynamics with continuous weights. 
Seungwoong Ha · Hawoong Jeong 🔗 


Provably Efficient Causal ModelBased Reinforcement Learning for EnvironmentAgnostic Generalization
(
Poster
)
link
In the sequential decision making setting, an agent aims to achieve systematic generalization over a large, possibly infinite, set of environments. Such environments are modeled as discrete Markov decision processes with both states and actions represented through a feature vector. The underlying structure of the environments allows the transition dynamics to be factored into two components: one that is environmentspecific and another one that is shared. Consider a set of environments that share the laws of motion as an illustrative example. In this setting, the agent can take a finite amount of rewardfree interactions from a subset of these environments. The agent then must be able to approximately solve any planning task defined over any environment in the original set, relying on the above interactions only. Can we design a provably efficient algorithm that achieves this ambitious goal of systematic generalization? In this paper, we give a partially positive answer to this question. First, we provide the first tractable formulation of systematic generalization by employing a causal viewpoint. Then, under specific structural assumptions, we provide a simple learning algorithm that allows us to guarantee any desired planning error up to an unavoidable suboptimality term, while showcasing a polynomial sample complexity. 
Mirco Mutti · Riccardo De Santi · Emanuele Rossi · Juan Calderon · Michael Bronstein · Marcello Restelli 🔗 