Remix.run Logo
discarded1023 4 hours ago

This was a big concern when I was an undergrad in the 1990s. I've since wondered if bunched implications / separation logic / separation algebras / ... [1] that emerged in the early 2000s has resolved this well enough. Opinions?

At least some of the problem was due to people unnecessarily restricting themselves to first-order logic for knowledge representation, as advocated by John McCarthy [2].

[1] https://en.wikipedia.org/wiki/Separation_logic

[2] see e.g. https://www-formal.stanford.edu/jmc/concepts.pdf

ian_j_butler an hour ago | parent | next [-]

> resolved this well enough. Opinions?

Not at all resolved. If anything it is worse than before as we begin to understand it better, and now there are different versions of it that cover representation, relevance, epistemics. Pivoting "away" from logic just relocates it again. Arguably the whole challenge of neurosymbolics is (still) getting a persistent sidecar for logic bolted onto something like a language model. We actually have fairly decent autoformalizers (!!) and we still can't make that work very well in general.

From one perspective, the frame problem is pretty closely related to the "binding problem", causal reasoning and ramifications in general, and relevance is central to all. We have good pure formalisms for relevance, epistemics, and do-logic too. But we can't get language models to drive them very well, and language models alone are terrible at trying to do this sort of thing natively (see distractor sensitivity, mediated causality and multi-hop reasoning with implicit bridges).

Neurosymbolics probably is the key, but until there's more traffic between old-school and new we're facing the same old problems. When/if there's real progress.. I think we'd know. It may or may not be AGI-complete but the improvements for things like long-horizon and truly out-of-distribution planning would probably be immediate, obvious, and jaw dropping

discarded1023 16 minutes ago | parent [-]

Thanks for the reply and pointers.

The intro to TFA> To most AI researchers, the frame problem is the challenge of representing the effects of action in logic without having to represent explicitly a large number of intuitively obvious non-effects. But to many philosophers, the AI researchers' frame problem is suggestive of wider epistemological issues. Is it possible, in principle, to limit the scope of the reasoning required to derive the consequences of an action? And, more generally, how do we account for our apparent ability to make decisions on the basis only of what is relevant to an ongoing situation without having explicitly to consider all that is not relevant?

Near as I can tell separation logic (suitably generalised/tamed/adapted to suit the system of interest/tools in use) addresses all these concerns. I'm not claiming it solves every last variant of the frame problem that anyone has ever considered; just that it seems to address the classical concerns about modularly specifying the effects of actions.

Take, for instance, the last question: separation logic models this by explicitly splitting the state (of the system of interest) into "relevant" and "not relevant" via separating conjunction (etc) and the suggestively-named "Frame" axiom takes care to preserve the "not relevant" part.

This partially addresses epistemics too, but I see that an action may affect things that I am not aware of. Though perhaps that is more of a modelling issue than a linguistic one.

I have no clue what does and does not work well with LLMs -- I'm just talking about explicit symbolic representation and (computer assisted/mechanised) reasoning; GOFAI but from a program logic perspective. Are you claiming that separation logic is unusable by LLMs? Or that it isn't helpful for capturing some essential aspects of framing in real-world problems?

Joker_vD 3 hours ago | parent | prev [-]

Heh. Reminds me of one of Lewis Carroll's sylogisms:

    Premise A: "No one, who means to go by the train and cannot get a conveyance, and has not enough time to walk to the station, can do without running";

    Premise B: "This party of tourists mean to go by the train and cannot get a conveyance, but they have plenty of time to walk to the station".

    Does the conclusion "This party of tourists need not run" hold?
It actually doesn't; here's a non-formulaic reason:

[Here is another opportunity, gentle Reader, for playing a trick on your innocent friend. Put the proposed Syllogism before him, and ask him what he thinks of the Conclusion.

He will reply “Why, it’s perfectly correct, of course! And if your precious Logic-book tells you it isn’t, don’t believe it! You don’t mean to tell me those tourists need to run? If I were one of them, and knew the Premisses to be true, I should be quite clear that I needn’t run—and I should walk!”

And you will reply “But suppose there was a mad bull behind you?”

And then your innocent friend will say “Hum! Ha! I must think that over a bit!”

You may then explain to him, as a convenient test of the soundness of a Syllogism, that, if circumstances can be invented which, without interfering with the truth of the Premisses, would make the Conclusion false, the Syllogism must be unsound.]