I've had similar issues with conversation memory in ChatGPT, whereby it will reference data in long-deleted conversations, independent of my settings or my having explicitly deleted stored memories.

The only fix has been to completely turn memory off and have it be given zero prior context - which is best, I don't want random prior unrelated conversations "polluting" future ones.

I don't understand the engineering rationale either, aside from the ethos of "move fast and break people"