You seem to intentionally be ignoring the original quote that any error may have caused them to be flung into space. This is patently false unless the one math error is pumping in hundreds of pounds more propellant and burning far longer than the scheduled burns. NASA would need to make a significant series of mistakes beyond orbital math for the "flung out into space" statement to be true.

They certainly could've gotten the return wrong but with a perigee of 119 miles they arent even in a stable orbit and likely could deorbit themselves using only rcs thrusters at apogee, or by just waiting a few orbits.

This is underselling the risks. On top of the many trajectories which push them into unrecoverable situations, leaving them stranded in orbit, there can be trajectories where the moon gives a gravity assist strong enough to fling the spacecraft into escape velocity, fulfilling the OP.

In fact, the trajectory they chose for this mission exploited the opposite effect to yield a free return without propellant expense.

In the modern day, the chance of a math error being the root cause behind this failure mode are vanishingly small, but minor burn execution mistakes that do not require hundreds of extra pounds of propellant are definitely plausible. They were extremely common in the early days of spaceflight and plagued most of the very first moon exploration attempts. Again, with modern RCS this is unlikely. But reentry is still incredibly tight and dangerous. Apollo famously had a +-1° safe entry corridor, and Orion is way heavier and coming in even faster. If their perigee was off they could’ve easily burned up or doubled their mission time, which they may not have been able to survive.