> In this setup, how would cross-core communication work, including cross-thread?

Communication via shared memory. DOS and friends run on core 0 more or less normally; core 1 runs your alternate programming. Allocate a shared ring buffer for messages in each direction, core 0 writes messages and updates the written index on one, reads messages and updates the read index on the other; vice version for core 1. Core 1 probably spins on checking the indexes if it doesn't have anything else to do.

Either allocate a big chunk of memory for core 1 work at startup, or let it message core 0 to allocate. Don't let core 1 call into DOS/BIOS syscalls directly or you'll have a big mess.

Next step in efficiency would be letting core 1 sleep when it's bored and use an inter-processor interrupt (IPI) to wake it up as needed.

Pretty much how its done in TempleOS