Java BigInteger is signed.

https://docs.oracle.com/javase/8/docs/api/java/math/BigInteg...

You deserialize with a leading 0x80 you don't get a working key. You can check the bit or you can just prepend a 0x00.