Back to publications

Continual adaptation · 2026

PACT

Recurrent Domain Incremental Adaptation of Time-Series Foundation Models

A frozen-host method that corrects distribution shift only where it appears in the embedding space, while keeping an explicit path through the original model for conditions that recur.

0.958 average macro-F1 across a four-condition bearing-fault stream with frozen Sundial
<0.05 relative forgetting on all five evaluated benchmarks
5% labeled calibration data per arriving condition, with no condition identity at test time
01

Motivation

Operating conditions change, and old ones come back.

A fault-diagnosis model trained under one machine load or speed will meet new conditions during its service life. The same happens across users in wearable sensing or subjects in physiological monitoring. Each new condition arrives with only a small labeled calibration set, and earlier conditions, including the source, keep recurring at inference without anyone saying which condition produced a sample.

Fine-tuning shared parameters recovers the new condition but damages earlier ones. Strong regularization limits that damage but also limits adaptation. PACT starts from a measurement on frozen TSFM embeddings: condition shift is highly non-uniform across the embedding space. Some local regions degrade substantially; others barely change, and a single global correction needlessly moves embeddings that were already classified correctly.

02

Method

An accumulating atlas around an immutable host.

Fix the charts

The pretrained encoder and source-trained linear head stay frozen. k-means on standardized source embeddings defines fixed nearest-centroid charts.

Fit local transports

For each arriving condition, PACT learns an identity-initialized rank-8 residual transport per chart from the small labeled calibration set, through the frozen head.

Freeze and append

The new transport bank is frozen and added to the atlas. Raw series and past class labels are discarded; only source and calibration embeddings are kept for routing.

Route without an ID

A per-chart MLP gate, refit after each arrival, softly mixes the source identity branch with all learned transports from the embedding alone.

A margin-based condition shows when the gated correction is too small to change the frozen model's prediction; identity routing reproduces it exactly. In practice this certifies 99.8% of initially correct source predictions on FD, 97.5% on HHAR, and 71.1% on UCI-HAR, and every certified prediction was preserved.

03

Performance

Strong adaptation with low forgetting across five benchmarks.

4 / 5

Best final average macro-F1

With frozen Sundial, PACT leads all 14 main-comparison baselines on FD, WISDM, HHAR, and UCI-HAR. On Sleep-EDF, LwF is ahead by 0.09 points.

+11.2 pts

Gain on fault diagnosis

Over the strongest baseline on FD, and +8.3 points on WISDM, while discarding past class labels after each update.

8 hosts

Backbone-agnostic

Works unchanged on classification-pretrained, forecasting-pretrained, and task-specific encoders. With Mantis, it reaches 0.985 on FD and 0.967 on WISDM.

Frozen Sundial backbone, averaged over three seeds and four condition orders. Higher AMF1 and lower relative forgetting (RF) are better.
Dataset Strongest baseline Baseline AMF1 PACT AMF1 PACT RF
FDDualCD0.84600.95750.0021
WISDMLwF0.79400.87740.0292
HHARDualCD0.82960.83670.0210
UCI-HARLwF0.73510.78640.0425
Sleep-EDFLwF0.61380.61290.0171

Baselines span test-time and source-free adaptation, fine-tuning, regularization, distillation, and domain-incremental methods (S-Prompts, DualCD). Against rehearsal, PACT beats embedding replay on FD and WISDM; replay methods lead on HHAR, UCI-HAR, and Sleep-EDF.

04

What the results show

Routing, not only low-rank adaptation, protects earlier conditions.

Ungated local transports improve plasticity but still apply unnecessary corrections. Adding the learned gate gives the best average macro-F1 and the lowest performance-weighted forgetting among adaptive variants on all five datasets. Rank-8 transports match or beat full-rank maps, and source k-means charts beat random-projection partitions at equal capacity.

Across eight frozen backbones the mechanism transfers unchanged, but final quality depends on the host: UCI-HAR ranges from 0.517 with PatchTST to 0.922 with Mantis. An embedding-space correction can only recover what the representation separates.

  • Growing memoryEach condition adds a transport bank, and gate refitting retains source and calibration embeddings, though not raw time series.
  • Overlapping conditionsRouting is harder on HHAR and UCI-HAR, where oracle routing leaves a visible gap, and gate refitting can change earlier predictions.
  • Hard chart assignmentNearest-centroid charts can be brittle near boundaries; adaptive refinement and soft assignment are natural next steps.
  • Covariate shift onlyThe label set and labeling function are assumed fixed; concept shift is out of scope.