I'd suggest going through one of the relevant reverse engineering courses (all free) in either pwn[.]college or/and OpenSecurityTraining2 .