The underlying memory is still binary, or were you proposing an entirely new computer architecture with ternary gates?

Not necessarily new - first ternary computer was around in 1959! https://en.wikipedia.org/wiki/Setun