> Sending a message between Actors can be just moving a pointer to a piece of shared memory.

No, you also need synchronization operations on the sending and the receiving end, even if you have a single sender and a single receiver. That's because message queues are implemented on top of shared memory – there's no way around this on general-purpose hardware.

Depends on your definition of synchronization operations. You certainly need memory fences, and possibly atomic operations. These may or may not have a performance cost.