Doing math without memorizing some basic arithmetic facts is like reading without knowing what the hundred most common words in the English language mean, and having to look them up every time you encounter one. Sure I guess you can do that, but… you definitely shouldn’t.