But then they could have just taken the LFS plugin and made it a core part of git, if that were the only problems.

If it didn't have those problems, it wouldn't really be git lfs, it would be something else.