My field is primarily audio/vision ML so I don't have any particular domain expertise, but I don't get how you can lump all "time-series" together in one big bucket. My naive understanding is that the way to improve the accuracy is to gain as much joint information as possible. Like:

> Known-future covariates. Some drivers of sensor behaviour are known in advance, such as production schedules, planned shutdowns, weather forecasts. Timestral can condition on them to improve forecast quality, so a scheduled load increase is expected rather than flagged as a surprise.

To me this should be the entire basis of the system, not a condition to improve forecast quality? Like if you're modelling electricity consumption, if all you have is a window of N previous samples, you're going to get completely beaten by a comparatively simple system that knows the time of day, that there's holidays coming up, the weather forecast, etc.

I would also think that these time-series forecasting models would gain from a sophisticated understanding at least the labels of the covariates? A 5M model has no idea what a "bank holiday" is, it just sees an arbitrary class go from 0 to 1, and won't have any useful inductive biases as to what it means or how to link them to other pieces of information.

E.g., wouldn't you predict €/MWh of electricity far more accurately with access to price of natural gas (itself influenced by many geopolitical factors), breakdown in composition of production of each energy source, predictions about roll-out (requiring a sophisticated LLM capable of understanding news and their implications), etc. What's the use in trying to predict it from 2 variables and a narrow window looking back at just 8192 or 32768 samples?

Or perhaps I misunderstand the use-case and this is more for extremely short term movements?

For some kinds of forecasting it doesn't need to be exactly accurate - eg say you are forecasting traffic for how to scale a load balancer, it can be a bit off it will just cost you a little and you can adjust after - a lot of feedback systems dont need full accuracy. You wouldnt use this to forecast whether to buy shares. It basically replaces various traditional curve fitting tools. If you need a really good forecast yes you are going to build a domain model. At least this is my understanding...

Forecaster of the Confounding Company here :)

You're right but, if the time series looks like music then you can apply your music domain knowledge. You don't have to only apply knowledge that's general across every domain, part of the magic of large neural nets is that you can do recognition and prediction at the same time thereby using (statistically) things known from training about how to identify a sequence and then also how to continue it.

Sure adding outside info helps even more, but with enough data it should become unnecessary (or so goes the theory).

I'm doing time series forecasting for a biological system. I haven't used these foundation models but my experience is that knowledge of how things are related is baked into what variables you put into the model (which can be more than raw data, you can make any derived product you think would be useful) but it's up to the model to figure out how they are related (if I knew exactly how they were related I wouldn't be training a forecasting model).

I'm sure someone who actually knows this stuff has a better easy to view it but my impression was that these time series foundation models aren't trained for a specific dataset but more they are trained on a procedure for how to look at datasets and make predictions. It doesn't seem to lump things into a bucket much more than any other model type (which also are just consistent procedures to look at datasets). It's not like you have radically different model types across different fields.

I interviewed for a company which used time series modelling to predict engine failure on ships. Though they didn't call it that, they called it just "AI" and weren't familiar with the time series literature at all. (Neither was I, but I did at least know that the time series literature was a thing.) They just used a bunch of sensors attached to the engine.

That seems like a scenario where there wouldn't be external sources of information capable of improving your predictions. It also seemed to me like a scenario where good old fashioned statistical methods would outperform LSTMs, which was what they used.

(I didn't get the job - I was offered it, but they rescinded their offer at the last moment, supposedly because their investors had insisted they should hire a web developer instead.)

I can see another use for time-series generative models; to create the right shape of noisy data.

To improve the robustness of machine learning models, artificial noise is often introduced into actual training data. Something like Timestral could simplify the process of generating it.

I appreciate this analysis, but I wonder, setting aside unpredictable current events for the moment, aren’t seasonal and scheduled events also covariates? eg forecasting baseball scores year round?

[dead]

[flagged]