Remix.run Logo
ehnto 5 hours ago

Fair warning, I have found local models and frontier models to be very bad at the specifics when it comes to cars.

Small differences like month and year model can impact oil capacity, oil weight and things like that, the details that matter quite a bit.

I found frontier models couldn't get things like what engine was in a 1994 Nissan Skyline, one of the more infamous and talked about cars on internet forums for decades, with dedicated fan databases that would have been scraped.

Questions like "what air filter do I need for my 1994 Suzuki Swift?" are hit and miss.

javier123454321 5 hours ago | parent | next [-]

This seems to be trained(? or referencing) on the specific cars' owner's manual.

efskap 2 hours ago | parent | next [-]

Yeah referencing is the way to go, as even finetuning probably captures style more than concrete facts. I know with large context windows we don't really RAG anymore, but for owner's manual lookup with a smaller model it seems ideal.

Something every LLM user ends up learning is that they're far better used as search and summarization tools than as knowledge databases in themselves.

4 hours ago | parent | prev [-]
[deleted]
alexandra_au an hour ago | parent | prev | next [-]

It's nothing that tool calling/feeding it the correct information can't solve

whalesalad an hour ago | parent | prev [-]

Just a few hours ago I gave ChatGPT my window sticker and the installation manual for a new suspension setup. I asked for new hardware that would typically be replaced during this install, like torque-to-yield bolts and fasteners. I also asked for new oil filters. I got a comprehensive grid of the exact part numbers needed in a nice dense table. sol 5.6 high is my daily driver.