Learning with Expected Signatures: Theory and Applications

Learning with Expected Signatures: Theory and Applications

Speakers: Lorenzo Lucchese, Mikko S. Pakkanen, Almut E. D. Veraart

Conference: ICML 2025

YouTube: https://slideslive.com/39044007

Overview

This article delves into the theoretical advancements and practical implications presented in the ICML 2025 talk "Learning with Expected Signatures: Theory and Applications." Delivered by Lorenzo Lucchese, in collaboration with his former PhD advisors Mikko S. Pakkanen and Almut E. D. Veraart, the presentation addresses critical challenges in applying expected signatures—a powerful non-parametric tool for characterizing probability distributions of paths—to real-world machine learning problems. While expected signatures offer a robust mathematical framework, their utility in practice is often hampered by the discrete, finite, and potentially dependent nature of observed data.

The core contribution of this work lies in providing rigorous theoretical guarantees for the convergence of empirically estimated expected signatures to their theoretical counterparts. The authors meticulously dissect the sources of error arising from discrete observations (infill error) and finite samples (estimation error), establishing conditions under which these errors diminish, leading to consistent and asymptotically normal estimators. Furthermore, they introduce a novel control variate-based correction to the empirical expected signature estimator, designed to reduce variance and improve estimation efficiency, particularly for processes exhibiting martingale-like properties.

This research significantly bridges the gap between the sophisticated theory of rough paths and the demands of practical machine learning applications. By validating the use of expected signatures with realistic data limitations, the work empowers practitioners in fields such as quantitative finance, signal processing, and time series analysis to leverage this powerful representation with greater confidence. The findings offer a solid theoretical foundation for building more robust and reliable models from complex data streams, thereby expanding the applicability of signature-based methodologies in data-driven decision-making.

Background

▶ Watch: Introduction and talk overview (0:00)

The concept of signatures originates from rough path theory, providing a robust, non-parametric way to represent continuous paths or data streams. For a well-behaved path X (e.g., bounded variation or smooth), its signature is defined as an infinite sequence of iterated integrals: S(X) = (1, ∫dX, ∫∫dXdX, ...). This representation possesses a remarkable property: its terms exhibit factorially decaying information. This means that the initial few terms of the signature capture the most significant information about the path, allowing for practical truncation to a finite-dimensional vector that still retains a rich, non-parametric summary of the original path. This characteristic has made signatures an attractive "out-of-the-box" solution in various non-parametric machine learning models when the input is a data stream and domain knowledge is limited.

Crucially, the utility of signatures extends beyond smooth paths. The theory of geometric rough paths generalizes the notion of integration, enabling the definition of signatures for much "rougher" processes, such as Brownian motions, Ito diffusions, and many Gaussian processes, which lack the regularity required for standard Riemann-Stieltjes or Lebesgue integration. This generalization vastly expands the applicability of signatures to a broader class of stochastic processes commonly encountered in scientific and engineering domains.

Building upon the concept of individual path signatures, the expected signature extends this idea to characterize the entire probability distribution of paths. If paths are sampled from a probability distribution on path space, the expected signature E[S(X)] is computed by taking the signature of each path realization and then averaging them. A pivotal result from 2016 established that the expected signature is characteristic for the law of the path, meaning that two probability distributions on path space are identical if and only if their expected signatures are identical. Like the signature of a single path, the expected signature also exhibits factorial decay in information, allowing for truncation to a finite vector that serves as a summary statistic for the entire path distribution. This property makes expected signatures highly valuable for developing non-parametric models that operate on collections of data streams, treating them as sample realizations from an underlying probability distribution.

Despite the theoretical elegance and power of expected signatures, their direct application in real-world scenarios faces several practical hurdles. Firstly, one rarely observes the full, continuous-time data distribution; instead, only a finite sub-sample of paths is available. Secondly, these paths are typically observed at discrete time points, rather than continuously. Thirdly, in many domains, particularly quantitative finance, data often comes as a single, long observation rather than many independent and identically distributed (IID) paths. To use such data, practitioners frequently "chop up" the long observation into shorter segments, implicitly assuming some form of stationarity or ergodicity. These discrepancies between theoretical assumptions and practical data availability necessitate a rigorous investigation into when the empirical expected signature—what can actually be computed from observed, discrete, finite, and potentially dependent data—can reliably approximate the true theoretical expected signature. This is precisely the problem space that Lucchese, Pakkanen, and Veraart address in their work.

Key Findings

▶ Watch: Signature: Factorial information decay, non-parametric representation (2:00)

The central contributions of this research revolve around providing a robust theoretical framework for the practical application of expected signatures, alongside proposing an improved estimation technique. The key findings can be summarized as follows:

1. Theoretical Guarantees for Empirical Expected Signatures: The talk establishes the conditions under which the empirical expected signature, computed from real-world, discretely observed, and finitely sampled data, consistently approximates the theoretical expected signature. This is a crucial validation for practitioners seeking to leverage this powerful tool.

1. Error Decomposition Methodology: A significant finding is the decomposition of the overall estimation error into two distinct components:

• Infill Error: Quantifies the discrepancy arising from observing paths at discrete time points instead of continuously.
• Estimation Error: Captures the error due to observing only a finite sub-sample of paths rather than the full probability distribution. This decomposition allows for a targeted analysis of each error source.

1. Convergence Results for Infill Error: For processes satisfying Holder regularity in a stochastic sense (e.g., alpha = 1/2 for Brownian motions, Ito processes, and many Gaussian processes) and an additional control on conditional increments, the infill error is shown to vanish in the L2 norm. This convergence is contingent on the observation partition being sufficiently refining, meaning the time steps become small enough.

1. Consistency and Asymptotic Normality for Long Span Asymptotics: Addressing the common scenario of non-IID data (e.g., chopping up a single long time series), the authors prove consistency for the empirical expected signature under assumptions of stationarity and ergodicity for the chopped-up increments, along with a refining observation partition. Furthermore, by adding conditions such as strong mixing and slightly faster convergence of the observation partition, they establish asymptotic normality for the estimator, providing confidence intervals and enabling hypothesis testing.

1. Novel Control Variate-Based Estimator Correction: A second major contribution is the proposal of a corrected estimator for the empirical expected signature. This correction leverages a control variate technique, particularly effective when the underlying process is a martingale. By introducing a mean-zero term (derived from Ito integrals) that is correlated with the original estimator, the proposed correction significantly reduces the variance of the estimator while preserving its bias. This offers a practical way to improve the efficiency of expected signature estimation.

These findings collectively provide a solid theoretical foundation for the practical deployment of expected signatures, addressing fundamental challenges posed by real-world data characteristics and offering a method for enhanced estimation performance.

Technical Deep Dive

▶ Watch: Expected signature: Characteristic for path law (4:00)

The technical core of this work lies in establishing rigorous mathematical conditions for the convergence of empirical expected signatures and in devising a variance-reducing estimator. Let's unpack the key concepts and methodologies presented.

The journey begins with the signature of a path X, denoted S(X). For a path X mapping from [0, T] to R^d, its signature is a sequence (1, S^1(X), S^2(X), ...) where S^k(X) represents the tensor of k-th order iterated integrals. For example, S^1(X)_i = ∫_0^T dX_i(t) and S^2(X)_{ij} = ∫_0^T ∫_0^s dX_i(t) dX_j(s). This sequence is known to be a faithful representation of the path, and its terms exhibit factorial decay in magnitude, allowing for truncation to a finite level N to form a practical feature vector. The talk emphasizes that this concept extends to rough paths, enabling its application to non-smooth processes like Brownian motion by using generalized integration theories (e.g., geometric rough path integral).

The expected signature, Φ = E[S(X)], is the central object of interest. It is the expectation of the signature of a path X drawn from a probability distribution P on path space. A key theoretical result from 2016 (Lévy et al.) states that Φ is characteristic for P, meaning it uniquely identifies the distribution P. In practice, however, we only observe a finite number of discrete paths. The goal is to understand how well the empirical expected signature, denoted Φ̂, approximates Φ.

The authors tackle this by decomposing the total estimation error into two parts. Let Φ̂ be the empirical expected signature computed from discrete observations of M paths, and let Φ_continuous_discrete be the expected signature computed if one had continuous observations of M paths from the true distribution. The error can be written as:

||Φ̂ - Φ|| <= ||Φ̂ - Φ_continuous_discrete|| + ||Φ_continuous_discrete - Φ||

The first term, ||Φ̂ - Φ_continuous_discrete||, quantifies the infill error arising from discrete time observations. The second term, ||Φ_continuous_discrete - Φ||, represents the estimation error from using a finite sample of M paths instead of the full distribution.

For the infill error, the paper provides convergence guarantees under specific regularity conditions. Assume the process X satisfies Holder regularity in a stochastic sense, specifically alpha = 1/2. This covers a broad class of processes including Brownian motions, Ito processes, and many Gaussian processes, which are characterized by bounded two-variation but potentially unbounded one-variation. Additionally, a condition A_delta is required, which controls the conditional increment of the process. For instance, for a martingale, this condition is trivially satisfied. Under these assumptions, and provided that the observation partition π (the set of discrete time points) is refining and its mesh goes to zero sufficiently fast, the infill error is shown to converge to zero in the L2 norm. This means that as observations become denser, the error introduced by discretization vanishes in mean-square.

For the estimation error, particularly relevant for non-IID data, the authors employ long span asymptotics. This scenario is common in finance, where a single long time series is "chopped up" into shorter segments to generate multiple "observations." To validate this approach, the work assumes that the process can be segmented into increments that are stationary and ergodic. Combined with the conditions on the observation partition, this leads to consistency: the empirical expected signature Φ̂ converges to the true Φ as the number of chopped-up segments increases. Further strengthening these results, if additional conditions like strong mixing are met and the observation partition converges slightly faster, the estimator Φ̂ is shown to be asymptotically normal. This is a powerful result, enabling the construction of confidence intervals and statistical inference based on empirical expected signatures.

Beyond theoretical convergence, the talk introduces a control variate-based correction to improve the empirical estimator. This technique is particularly effective when the underlying process X is a martingale. The signature is typically defined using Stratonovich integrals. For a martingale, the corresponding Ito integral has a mean of zero. The core idea is to augment the empirical Stratonovich signature estimator with a term involving the empirical Ito integral. Specifically, if S_Stratonovich is the target and S_Ito is the control variate (with E[S_Ito] = 0), the corrected estimator takes the form:

Φ̂_corrected = Φ̂_Stratonovich - c * Φ̂_Ito

where c is a coefficient chosen to minimize variance, typically c = Cov(Φ̂_Stratonovich, Φ̂_Ito) / Var(Φ̂_Ito). Since Φ̂_Ito has zero mean, subtracting a scaled version of it does not introduce bias, but if Φ̂_Stratonovich and Φ̂_Ito are highly correlated, it can significantly reduce the variance of the estimator. The authors note that c must be estimated from data, adding a practical consideration. They also highlight that this correction can still yield improvements due to a favorable bias-variance trade-off even if the process is only "close" to a martingale. This novel correction offers a practical avenue for improving the efficiency and reliability of expected signature estimation in relevant contexts.

Experimental Setup & Results

▶ Watch: Decomposing expected signature estimation error (6:00)

The talk primarily focused on the theoretical underpinnings and the proposed estimator correction, with detailed experimental setups and quantitative results from the paper only briefly touched upon during the Q&A segment. The speaker acknowledged that the full paper contains comprehensive experimental validations.

During the Q&A, a question was raised regarding the number of samples required for a good approximation of the expected signature in practice, particularly in finance where processes can be non-stationary and often only a single long path is observed. In response, Lorenzo Lucchese mentioned that in some of their experiments, they found that "orders of tens" to "maybe 20, 50 or 100 paths" were sufficient to achieve a desirable level of performance. He clarified that these experiments were conducted using "all synthetic data," specifically mentioning a "rough volatility experiment."

For scenarios where only a single long observation is available, as is common in finance, the approach involves "chopping it up into say maybe daily realizations." This could yield, for example, "240 daily realizations" from a year's worth of data, providing a sufficient number of "paths" for estimation. The speaker emphasized that while the exact number depends on various assumptions and the desired accuracy, these empirical observations suggest that expected signatures can be practically estimated with a relatively modest number of observations, even when derived from a single long time series.

The talk did not delve into specific model architectures, hardware, or detailed comparative benchmarks but rather concentrated on the theoretical validation and estimator improvements. The emphasis was on establishing the statistical properties of the empirical expected signature under realistic data constraints rather than showcasing specific application performance metrics.

Practical Implications

▶ Watch: Convergence conditions for empirical expected signatures (7:00)

The theoretical results and the proposed estimator correction presented in this talk carry significant practical implications for machine learning practitioners, infrastructure teams, model builders, and deployers working with time series and sequential data.

Firstly, and perhaps most importantly, this work provides a much-needed theoretical validation for the use of expected signatures in real-world ML applications. Before this research, practitioners might have hesitated to apply expected signatures due to the stark contrast between the continuous, infinite-sample assumptions of the theory and the discrete, finite, and often dependent nature of available data. By rigorously demonstrating the convergence of empirical expected signatures under realistic conditions (discrete observations, finite samples, non-IID data), the authors instill confidence in the reliability and statistical soundness of models built upon this representation. This means that features derived from expected signatures can be trusted as consistent and asymptotically normal estimators of the underlying path distribution.

The decomposition of error into infill error and estimation error offers a clear roadmap for understanding and mitigating potential issues. For instance, infrastructure teams collecting data can prioritize denser sampling (reducing infill error) if their processes exhibit significant roughness. Conversely, if the process is smoother, less frequent sampling might suffice without sacrificing too much accuracy. Model builders can now better assess the quality of their expected signature features based on the characteristics of their data collection (e.g., sampling frequency, sample size, independence assumptions).

The long span asymptotics results are particularly impactful for domains like quantitative finance, where observing multiple IID paths is often impossible. The ability to "chop up" a single, long time series into segments and still obtain consistent and asymptotically normal estimates of the expected signature is a game-changer. This validates a common heuristic practice and provides a principled way to extract rich, non-parametric features from extensive historical data. It means that models can be trained on features derived from, say, daily or hourly segments of years of financial data, with theoretical guarantees on the quality of these features, provided the underlying process exhibits sufficient stationarity and ergodicity over the chopping window.

The proposed control variate-based correction offers a direct method for improving estimation efficiency. For processes that are martingales or "close enough" to martingales (a common idealization or approximation in finance), this correction can significantly reduce the variance of the expected signature estimator. Reduced variance translates to more stable and reliable feature representations, leading to better performing and more robust ML models. Practitioners can implement this correction to get more bang for their buck from their limited data, especially when dealing with noisy or high-volatility processes. The trade-off here is the need to estimate the correlation coefficient C from data, which adds a practical step but is generally feasible.

However, there are practical considerations and limitations. The theoretical guarantees rely on specific assumptions about the underlying stochastic process, such as Holder regularity, control on conditional increments, stationarity, ergodicity, and strong mixing. Practitioners must exercise due diligence to ensure these assumptions are reasonably met by their data. For instance, if a financial time series exhibits strong non-stationarities or regime shifts, simply chopping it up might violate the stationarity assumption, potentially compromising the validity of the results. The computational cost of computing signatures, while mitigated by truncation, can still be substantial for very high-order terms or very long paths, necessitating careful consideration of the truncation level. Furthermore, the Q&A highlighted the ongoing research into alternative "bases" or representations (like the log signature) that might offer faster convergence or computational advantages, suggesting that while signatures are powerful, they are part of an evolving toolkit.

In essence, this work elevates expected signatures from a theoretical curiosity to a practically validated and enhanced tool for machine learning with sequential data, empowering a broader range of applications and fostering more confident model development.

Key Takeaways

• Bridging Theory and Practice: The research provides crucial theoretical guarantees for using empirical expected signatures in real-world ML, addressing challenges posed by discrete observations, finite samples, and non-IID data.
• Error Decomposition: The total estimation error is rigorously decomposed into infill error (from discrete sampling) and estimation error (from finite samples), allowing for targeted analysis and mitigation.
• Robust Convergence: Under conditions of Holder regularity (e.g., alpha=1/2 for Ito/Brownian processes) and sufficient partition refinement, the infill error converges in L2 norm. For non-IID data (long span), consistency and asymptotic normality are established under stationarity, ergodicity, and strong mixing assumptions.
• Variance Reduction with Control Variates: A novel control variate-based correction is proposed for the empirical expected signature, significantly reducing its variance, particularly beneficial for processes that are martingales or martingale-like.
• Enhanced Practicality for Time Series: The findings validate the practice of "chopping up" long, dependent time series into segments for expected signature estimation, making the method highly applicable in domains like quantitative finance and signal processing.
• Modest Sample Requirements: Empirical observations suggest that "orders of tens" to "hundreds" of paths can be sufficient for desirable performance, even when derived from a single long observation.

About the Speaker(s)

The talk was presented by Lorenzo Lucchese, who conducted this research as joint work with his former PhD advisors, Mikko S. Pakkanen and Almut E. D. Veraart. Lorenzo Lucchese mentioned his background in stochastic analysis and expressed interest in the quantitative finance industry, indicating a practical orientation to his research. Mikko S. Pakkanen and Almut E. D. Veraart are acknowledged as key collaborators and advisors on this work, reflecting their expertise in the field. The collective background of the speakers in stochastic analysis underpins the rigorous mathematical approach taken in the research.