Data-Centric Pretraining for Time-Series Foundation Models

A line of work on the data behind time-series foundation models: how to measure what a pretraining corpus actually contributes, select the samples that matter, and synthesize new data to close specific coverage gaps — improving forecasting while shrinking the corpus.

Problem

Pretraining mixtures for time-series foundation models are largely chosen by intuition. There is little principled signal on which samples improve generalization, which are redundant, and where the corpus is thin.

Approach

Characterize time series in a low-dimensional space built from temporal, statistical, and spectral attributes together with foundation-model embeddings. Use neighborhood structure in that space both to select influential, diverse pretraining samples and to quantify gaps — then fill those gaps with targeted synthetic data generated through a frequency-domain transform.

Results

Data selection reaches comparable forecasting performance with a substantially smaller pretraining corpus (ICASSP 2026); targeted synthetic generation yields in- and out-of-distribution improvements across Mixer, Transformer, and State-Space architectures (CODS 2026, under review).