Remix.run Logo
aranibatta 3 days ago

Congrats on launching! Curious how early the Mastra team thinks people should be thinking about evals and setting up a pipeline for them.

calcsam 3 days ago | parent [-]

We tend to recommend folks spend a few hours writing evals after they spend a couple weeks prototyping. Then they get a sense of how valuable evals are for their use-case.

We think about evals a bit like perf monitoring -- it's good to have RUM but also good to have some synthetic stuff in your CI. So if you do find them valuable, useful to do both.