I agree with the above comment on a broad normative (what is good) take: on a forum for humans, yes, please, bring your human self. But there is a lot of room for variety, choice, even self-expression in the be your human self part! Some might prefer using the Encyclopedia Brittanica to supplement an imperfect memory. Others DuckDuckGo. Some might bounce their ideas off friends. Or (gasp) an LLM. Do any of these make the person less human? Nope.

Of course, there are many ways to be more and less intellectually honest, and there is a lot to read on this, such as [1].

Now, on the descriptive / positive claims (what exists), I want to weigh in:

> LLMs are an autocomplete engine.

Like all metaphors, we should ask the "what is the metaphor useful for?" rather than arguing the metaphor itself, which can easily degenerate into a definitional morass. Instead, we should discuss the behavior, something we can observe.

> [LLMs] aren't curious.

Defined how? If put aside questions of consciousness and focus on measuring what we can observe, what do we see? (Think Turing [2], not Chalmers [3].) To what degree are the outputs of modern AI systems distinguishable from the outputs of a human typing on a keyboard?

> LLMs CANNOT provide unique objectivity...

Compared to what? Humans? The phrasing unique objectivity would need to be pinned down more first. In any case, modern researchers aren't interested in vanilla LLMs; they are interested in hybrid systems and/or what comes next.

Intelligence is the core concept here. As I implied in the previous paragraph, intelligence (once we pick a working definition) is something we can measure. Intelligence does not have to be human or even biological. There is no physics-based reason an AI can't one day match and exceed human intelligence.*

> or offer unknown arguments ...

This is the kind of statement that humans are really good at wiggling out of. We move the goalposts. So I'll give one goalpost: modern AI systems have indeed made novel contributions to mathematics. [4]

> because they can only use their own training data, based on existing objectivity and arguments, to write a response.

Yes, when any ML system operates outside of its training distribution, we lose formal guarantees of performance; this becomes sort of an empirical question. It is a fascinating complicated area to research.

Personally, I wouldn't bet against LLMs as being a valuable and capable component in hybrid AI systems for many years. Experts have interesting guesses on where the next "big" innovations are likely to come from.

[1]: Tversky, A., & Kahneman, D. (1974). Judgment under Uncertainty: Heuristics and Biases: Biases in judgments reveal some heuristics of thinking under uncertainty. science, 185(4157), 1124-1131.

[2]: The Turing Test : Stanford Encyclopedia of Philosophy : https://plato.stanford.edu/entries/turing-test/

[3]: The Hard Problem of Consciousness : Internet Encyclopedia of Philosophy : https://iep.utm.edu/hard-problem-of-conciousness/

[4]: FunSearch: Making new discoveries in mathematical sciences using Large Language Models : Alhussein Fawzi and Bernardino Romera Paredes : https://deepmind.google/blog/funsearch-making-new-discoverie...

* Taking materialism as a given.