Does it still ultimately call into gradle to perform the build?

Yes, since we need Gradle dependencies in order to build rich UI with AndroidX or Material Design. But if you're interested in a minimal approach without Gradle, check out the example by @purpln here: https://github.com/purpln/android-example