Learn uncertainty
A roughly 10K-parameter causal transformer predicts the next quantized value. After pretraining, it is frozen and used only to estimate conditional entropy.
Time-series transformers · 2026
EntroPE
Entropy Guided Dynamic Patch Segmentation for Time Series Transformers
A dynamic patching framework that places boundaries where the future becomes hard to predict, preserving coherent temporal patterns instead of cutting every sequence at fixed intervals.
9.7%
average MSE reduction against PatchTST across six forecasting benchmarks
77.13%
average classification accuracy across 10 UEA datasets
87.72%
average anomaly-detection F1 across five benchmarks
01
Motivation
Fixed windows ignore where patterns actually change.
Patch-based transformers make long sequences tractable by grouping nearby time points into tokens. Most methods use a fixed patch length, however, so their boundaries are determined by the starting index rather than the signal. A trend reversal, volatility burst, or seasonal transition can be split across tokens.
This fragmentation weakens the local representation and can create a train-inference mismatch: training exposes the model to many temporal offsets, while inference extracts patches from predetermined positions. The patcher should follow the predictive structure of the sequence, not the clock.
02
Method
Dynamic boundaries with fixed-size representations.
Detect transitions
Sample-adaptive quantiles flag positions with both high entropy and a sharp entropy change. Consecutive cuts are suppressed to form non-overlapping variable-length patches.
Encode each patch
Pooling initializes a fixed-size patch token; cross-attention then lets that token query only the time points inside its own patch, preserving local detail.
Fuse local and global
A global transformer models relations between patches. A fusion decoder sends global patch context back to time-point representations before the task head predicts.
Quantization is used only by the entropy model. Forecasting, classification, and anomaly heads operate on continuous representations, avoiding a quantized output range.
03
Performance
Better patches improve three different tasks.
20%
Lower ETTh1 MSE
Average MSE across four forecast horizons falls from 0.516 with PatchTST to 0.416 with EntroPE.
+6.0 pts
Classification over PatchTST
UEA average accuracy is 77.13%, compared with 71.14% for PatchTST and 76.03% for TSLANet.
1-3%
Dynamic-patching overhead
Measured epoch-time overhead stays small across the six forecasting datasets, while patch count remains adaptive.
| Dataset | PatchTST | EntroPE | Relative MSE reduction |
|---|---|---|---|
| ETTh1 | 0.516 | 0.416 | 19% |
| ETTh2 | 0.391 | 0.366 | 6% |
| ETTm1 | 0.406 | 0.378 | 7% |
| ETTm2 | 0.290 | 0.286 | 1% |
| Weather | 0.265 | 0.242 | 9% |
| Electricity | 0.216 | 0.182 | 16% |
The 9.7% headline is the mean of the six dataset-level relative MSE reductions shown above; it is not an absolute error-point reduction.
04
What the results show
Boundary placement is a useful modeling decision, not preprocessing trivia.
Component ablations degrade progressively as dynamic patching, the adaptive encoder, and the fusion decoder are removed. Dynamic patches also outperform fixed lengths in most evaluated forecasting settings, while the entropy percentile remains stable over a broad 15%-95% range.
The threshold doubles as an efficiency control: lowering it yields finer segmentation and more computation; raising it creates fewer patches. This makes the accuracy-cost tradeoff explicit rather than hidden in a fixed patch size.
- Predictive, not semantic, boundariesAn entropy peak signals uncertainty; it does not guarantee a human-named event boundary.
- Channel-independent backboneEach variable is patched and encoded independently before task-level aggregation.
- Frozen boundary modelThe entropy model is pretrained once, then held fixed while downstream task components learn.