NeurIPS 2026 · Main Conference · Sydney

EntroPE

Entropy Guided Dynamic Patch Segmentation for Time Series Transformers

*Corresponding author

Interactive sketch on a synthetic signal with a toy entropy estimator. It shows the mechanism; the numbers are for this window only. The paper measures 2.8× lower within-patch variance across 6 real datasets.

TL;DR. Patch-based transformers chop time series into fixed windows that often cut straight through a transition. EntroPE asks a tiny frozen model “how unpredictable is the next step?” and starts a new patch wherever the answer is “very”. Patches then follow the signal's own structure, which improves forecasting, classification, and anomaly detection while using far fewer parameters.

3.2%

lower average MSE than PatchTST across 8 forecasting datasets

90.9%

fewer parameters than PatchTST (and 78.5% fewer MACs)

2.8×

more internally homogeneous patches than fixed patching

97.2%

of detected boundaries fall at high-uncertainty positions

Abstract

Patch-based transformers have emerged as efficient, improved architectures for long-horizon time series modeling. Yet existing approaches rely on temporally-agnostic patch construction, in which arbitrary starting positions and fixed lengths fracture temporal coherence by splitting natural transitions across patch boundaries. This naive segmentation often disrupts short-term dependencies and weakens representation learning.

We propose a novel Entropy-Guided Dynamic Patch Encoder (EntroPE), a temporally informed framework that dynamically detects transition points via conditional entropy and dynamically places patch boundaries. This preserves temporal structure while retaining the computational benefits of patching. EntroPE consists of 2 key modules, namely an Entropy-Guided Dynamic Patcher (EDP) that applies information-theoretic criteria to locate natural temporal shifts and determine patch boundaries, and an Adaptive Patch Encoder (APE) that employs pooling and cross-attention to capture intra-patch dependencies and produce fixed-size latent representations.

Extensive experiments on long-term forecasting, classification, and anomaly detection demonstrate that the proposed method improves both accuracy and efficiency, establishing entropy-guided dynamic patching as a promising new paradigm for time series modeling.

Why fixed patches fall short

Boundaries should sit where the past stops being enough to predict the future.

Fixed-length patching treats time as homogeneous. This causes two problems. First, fragmentation: a trend change or seasonal transition gets split across patches, separating related time points. Second, a train–inference mismatch: training sees patches at many offsets relative to the underlying patterns, while inference uses fixed positions.

EntroPE places boundaries at moments of high conditional entropy of the next observation given its history. Regime changes land on patch boundaries, while patch interiors stay predictable and coherent.

Top: fixed-length patching with boundaries cutting through peaks. Bottom: temporally informed patching with boundaries at entropy peaks, and a time series coloured by entropy.
Figure 1. (A) Fixed-length patches fragment coherent patterns. (B) EntroPE puts boundaries at entropy peaks.

Method

A lightweight entropy model decides where to cut; the rest of the network learns from variable-length patches.

0

Frozen entropy model

A 2-layer GPT-style transformer (about 16K parameters) learns next-token prediction on quantized values, then is frozen. Quantization is used only to estimate entropy.

1

Entropy-guided dynamic patcher

A new patch starts after positions whose entropy, or rise in entropy, exceeds a per-sample α-quantile. Boundaries can be precomputed and add about 0.1–1.0 s per training epoch.

2

Adaptive patch encoder

Max-pooling initialises each patch embedding; cross-attention over the patch's own time points refines it into a fixed-size vector, whatever the patch length.

3

Global transformer and fusion decoder

A transformer models dependencies between patches, and the fusion decoder injects that context back into token states. Predictions stay continuous.

EntroPE architecture: channel independence, tokenizer, entropy-guided patcher, adaptive patch encoder, global transformer, fusion decoder, and linear head, with detail panels for the patcher, encoder, and decoder.
Figure 2. EntroPE architecture. (A) The EDP uses a frozen causal transformer to estimate next-token entropy and select boundaries. (B) The APE turns variable-length patches into fixed-size embeddings. (C) The fusion decoder combines global patch context with local time-point representations.

Results

One backbone for forecasting, classification, and anomaly detection; only the task head changes.

Long-term forecasting

Average MSE over 8 datasets and 4 horizons (input length 96). Further left is better.

  • EntroPE0.330
  • CALF0.333
  • TimeMosaic0.336
  • iTransformer0.337
  • FilterTS0.339
  • TSLANet0.340
  • TimeMixer0.341
  • PatchTST0.341
  • HDMixer0.353
  • FEDformer0.381
  • Time-FFM0.383
  • TimesNet0.388
  • TimeBase0.401
  • DLinear0.403

EntroPE is best on 6 of 9 columns of Table 1, including the overall average. It stays competitive with CALF (~180M parameters) and TimeMosaic (~8M) using ≤0.8M.

Classification (14 UEA datasets)

Average accuracy (%).

  • EntroPE76.01
  • TSLANet72.65
  • TS-TCC70.98
  • PatchTST68.53
  • GPT4TS66.72
  • ROCKET65.18

Anomaly detection (5 benchmarks)

Average F1 (%) on SMD, MSL, SMAP, SWaT, and PSM.

  • EntroPE88.47
  • TSLANet87.54
  • GPT4TS86.72
  • TimesNet85.24
  • FEDformer84.97
  • LightTS84.23
  • ETSformer82.87
  • PatchTST82.79
  • DLinear82.46
1st of 7

Entropy is the best boundary signal tested. With the architecture held fixed, conditional predictive entropy gives lower MSE than variance change-point, spectral shift, CUSUM, local difference, empirical Shannon entropy, and static boundaries on ETTm1, ETTh2, and Weather. Removing the dynamic patcher costs more than removing the encoder or decoder.

Entropy-guided patch placement on example windows from ETTm1, Weather, and Solar at input lengths 96 and 336, with signals above and entropy curves below.
Figure 8. Entropy-guided patches on real windows from ETTm1, Weather, and Solar. Boundaries cluster around bursts of activity and stay sparse across long, flat, low-information stretches, so computation goes where the information is.
Bar charts of test MSE for seven boundary signals on ETTm1, ETTh2, and Weather; conditional predictive entropy is lowest in each.
Figure 4. Boundary-signal ablation with a matched architecture: only the boundary rule changes.
ETT-family average MSE versus MACs per sample on a log scale, with bubble size showing parameters; EntroPE sits at the low-error end of the Pareto frontier.
Figure 6. ETT-family average MSE versus counted MACs per sample (~42M for EntroPE). Bubble size shows parameter count.
Test MSE versus inference latency on ETTm1, Weather, and Solar for fixed patch lengths and for EntroPE's dynamic patching.
Figure 7. Test MSE versus inference latency for dynamic patching and fixed patch lengths 1–12, including entropy estimation time.

Limitations

BibTeX

@inproceedings{abeywickrama2026entrope,
  title     = {Entropy Guided Dynamic Patch Segmentation
               for Time Series Transformers},
  author    = {Abeywickrama, Sachith and Eldele, Emadeldeen and Wu, Min
               and Li, Xiaoli and Yuen, Chau},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2026}
}