It's not as bad as you say. If you search in the Phone app, a single letter search returns contacts with a name field (and then a company field, and then a notes field) starting with that letter. It works reasonably well.

The Contacts app is worse and returns anything with a string anywhere in the contact details starting with that letter.