Agreed. The test I run is "Can it simplify a concept I know in a matter of sentences/words, better than I can?" on a subject area that requires a lot of nuance.
And to my disappointment, the LLMs never come close. They dump paragraphs upon paragraphs. I actually want to see the LLMs surprise me. But they disappoint me for what Im specifically looking for.
> Summarize a technique for type erasure as is done in C++. Assume that your audience is familiar with all of C++'s features, but not with the technique of type erasure. The briefer the summary, the better, but don't omit any essential pieces or use vague language.
ChatGPT Plus generated a lot of text and code. Then I said:
> Now please summarize using only a few sentences.
and it replied:
> Type erasure wraps any concrete type `T` in a uniform container that stores a `void*` to the object plus a pointer to a type-specific static table of function pointers (the “vtable”). Each `T` provides its own table implementing required operations (destroy, copy/move, and any interface methods), and the container dispatches all behavior through that table. This lets unrelated types share a runtime interface without a common base class, with lifetime and semantics fully defined by the per-type table.
A thing that eventually brute forces its way to a solution is not a) demonstrating understanding b) intelligent.
Im not sure why these basic things have to be uttered. I suppose its because many here don't really work on stuff that requires this level of thinking.
Reminds me of the same delusional people that think LLMs are the solution to education et al when they completely miss the point of what the real problems in education actually are.
The point is you shouldn't have to re-prompt multiple times because a thing that 'understands' undergoes expressing the thing in a broad way and then abstracts away what matters whilst minimizing loss of important details.
The reality is LLMs are not intelligent, in any way shape or form.
And to my disappointment, the LLMs never come close. They dump paragraphs upon paragraphs. I actually want to see the LLMs surprise me. But they disappoint me for what Im specifically looking for.