I'm not a Swift user, but I can tell you from C++ experience that this logic doesn't mitigate a complex programming language.

* If you're in a team (or reading code in a third-party repo) then you need to know whatever features are used in that code, even if they're not in "your" subset of the language.

* Different codebases using different subsets of the language can feel quite different, which is annoying even if you know all the features used in them.

* Even if you're writing code entirely on your own, you still end up needing to learn about more language features than you need to for your code in order that you can make an informed decision about what goes in "your" subset.