It cannot write some kind of short texts either.

The experiment was: given existing Experience section of Senior Software Engineer CV (manually and carefully written), write 2-4 lines of high pitch About [me].

gpt-sol-xhigh just could not do it, making a complete AI slop mess.

It became substantially better when I asked to get a sample of real Senior/Principal engineers CVs, that it would be able to connect with meaningful online presence/contributions of their authors, and draw ideas from that.

Nevertheless, the remaining issues were critical, with their classes spanning:

- word for word repetition;

- tautology (phrases mapping to the same semantic entity);

- category mistakes (combining apples with oranges);

- faulty composition of generalised and concrete terms.

Before throwing it away I decided to give it a try and asked for strict prompt following, setting low logical errors threshold, and eventually providing a formal proof that it complied.

It took spaCy, doing NER and dependency parsing; then I suggested adding stanza for constituency parsing.

After slapping together the artefacts of analysis and thinking a bit, it produced great phrase (to my taste).

-

Then when I asked to apply the framework and improve bullet points in some experience block, it did 2 of 4 well, then miserably broke down; I’m not sure if it’s harness issue (codex) or fundamental model restriction.

-

So no, without substantial investment in steering, SOTA AI doesn’t perform even remotely close to a human, in complex reasoning.