| ▲ | lovelearning 9 hours ago | |
My understanding is that the synthetic training data helps capture abstract time-series patterns that are common in all domains. As they say in appendix 8: > We create the synthetic data to reflect common time-series patterns using traditional statistical models. We start with four simple times series patterns: > • Piece-wise linear trends (I), where the number of the piece-wise linear components is randomly chosen between 2 and 8. > • ARMA(p, q) (II), where 1 ≤ p, q ≤ 8 and the corresponding coefficients are generated from either a multivariate Gaussian or a uniform, then normalized. > • Seasonal patterns. In particular we create the sine (III) and the cosine (IV) waves of different random periods between 4 and max context length / 2 time-points and time delays. If there were no such underlying patterns in the class of all time-series data, then even the idea of traditional time-series models would be fundamentally misplaced. And since this is a transformer model, it also looks for patterns in the problem-specific input data at inference time, just like how the input context to an LLM influences its output's relevance. | ||