I absolutely love audiobookshelf for audiobooks. And I currently use it for my epubs, but it does not handle that particularly well. It doesn't save progress consistently and the UI for page flipping leaves a lot to be desired. Definitely going to check out how OP's library works

You should check out Storyteller. It aligns ebooks with audiobooks to create EPUB3s with media overlays so you can do immersive reading. It also gives you a place to organize your audiobooks and epub library.