lanceDB and underlying lance works well as append only right? Wont you have to do index maintenance often for a chat access pattern?

Good question. This is targeted more at smaller chat server use cases that put the admins in the drivers seat. Compaction operations and rebuilding indexes can happen during a maintenance window.

I plan on scaling up the number of messages the bot system generates to see where the breaking point is.