> For sure. You cannot have "only higher level thoughts" without doing lower level work

What do you mean? I think people routinely think about things at a very high level with almost no understanding of the lower levels. How many people use a computer each day and reason about them at a very high level while knowing nothing of capacitors, logic gates, or programming languages?

▲

customguy 2 hours ago | parent | next [-]

How many people struggle with their computer, or get scammed, because to them it's just icons on a screen, with not even the concept of a process, memory vs. disk, or anything? How much money is lost each year because someone doesn't know what an URL is?

▲

throw10920 3 hours ago | parent | prev [-]

I think they didn't phrase it precisely, but my guess is the underlying idea is actually "high-level software architecture doesn't have a clear abstraction layer you can use to separate it from low-level coding (unlike logic gates, the CPU's ISA, the kernel API, etc), and so delegating the latter leads to delegating the former".

▲

skinfaxi 2 hours ago | parent [-]

That makes sense but I'm still not so sure—we have things like software architecture patterns that can be discussed at a high level without knowing the intricacies. Like you can be aware of load balancing and even use it but be unaware of how load balancing might work algorithmically.

Let's consider even the original example. > You cannot remove facts like 'paris is capital of france' from llm and have it just retain 'high level thoughts' like 'countries have capitals that you can look up'

Wouldn't the knowledge that countries have capitals precede the knowledge that Paris is the capital of France?

This says nothing about the accuracy of our own models based on these abstractions that lack the lower-level understanding.

	▲	throw10920 2 hours ago \| parent \| next [-]
		> we have things like software architecture patterns that can be discussed at a high level without knowing the intricacies I think the counterargument would be "you can't teach people architecture alone and get good architects". I've observed this myself in "systems engineers" whose job is to connect boxes together without understanding how the boxes work. They, invariably, design ridiculous architectures on their own and need to basically find a domain expert to route their opinions through to come up with anything sane.
	▲	skydhash 30 minutes ago \| parent \| prev [-]
		> we have things like software architecture patterns that can be discussed at a high level without knowing the intricacies. Like you can be aware of load balancing and even use it but be unaware of how load balancing might work algorithmically. It can be fine as a “user”, but not really as a “designer”. Because discussion about possible solutions is a matter of tradeoffs and tweaking of parameters, not slinging words around. Abstractions are not appliances that are plug and play. They’re often full of parameters that dictate their usefulness and costs, and not understanding those parameters is just roleplaying.